[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:39:12 dist-test-slave-4d5t krb5kdc[23738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:39:12 dist-test-slave-4d5t krb5kdc[23738](info): set up 2 sockets
Sep 26 20:39:12 dist-test-slave-4d5t krb5kdc[23738](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 20:39:13.821331 23731 mini_kdc.cc:121] Time spent starting KDC: real 2.109s user 0.003s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:39:13 dist-test-slave-4d5t krb5kdc[23738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:39:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:39:14.020743 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:37893
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:37893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:14.392824 23751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:14.393317 23751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:14.393599 23751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:14.402565 23751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:14.402696 23751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:14.402776 23751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:14.402875 23751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:14.402969 23751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:39:14.414319 23751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:37893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:37893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:14.416424 23751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:14.418044 23751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:14.434491 23756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:14.434661 23759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:14.434963 23757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:14.437002 23751 server_base.cc:1034] running on GCE node
I20260926 20:39:14.438021 23751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:14.439625 23751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:14.440907 23751 hybrid_clock.cc:648] HybridClock initialized: now 1790455154440815 us; error 43 us; skew 500 ppm
I20260926 20:39:14.441332 23751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:14 dist-test-slave-4d5t krb5kdc[23738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455154, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:14.444990 23751 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:39:14.446944 23751 webserver.cc:460] Webserver started at http://127.23.44.254:37901/ using document root <none> and password file <none>
I20260926 20:39:14.447721 23751 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:14.447881 23751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:14.448208 23751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:14.451839 23751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "04af2f7e17b2438fbbe38565c55c4988"
format_stamp: "Formatted at 2026-09-26 20:39:14 on dist-test-slave-4d5t"
server_key: "148c811e3b4b9e5fa5e0a2d99b67c4fb"
server_key_iv: "a005737a22e383484e82c12e3ece4e9d"
server_key_version: "encryptionkey@0"
I20260926 20:39:14.452780 23751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "04af2f7e17b2438fbbe38565c55c4988"
format_stamp: "Formatted at 2026-09-26 20:39:14 on dist-test-slave-4d5t"
server_key: "148c811e3b4b9e5fa5e0a2d99b67c4fb"
server_key_iv: "a005737a22e383484e82c12e3ece4e9d"
server_key_version: "encryptionkey@0"
I20260926 20:39:14.458587 23751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:39:14.463850 23766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:14.465979 23751 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260926 20:39:14.466230 23751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "04af2f7e17b2438fbbe38565c55c4988"
format_stamp: "Formatted at 2026-09-26 20:39:14 on dist-test-slave-4d5t"
server_key: "148c811e3b4b9e5fa5e0a2d99b67c4fb"
server_key_iv: "a005737a22e383484e82c12e3ece4e9d"
server_key_version: "encryptionkey@0"
I20260926 20:39:14.466475 23751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:14.486811 23751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:14.491442 23751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:14.491745 23751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:14.529487 23751 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:37893
I20260926 20:39:14.529536 23817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:37893 every 8 connection(s)
I20260926 20:39:14.530946 23751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:14.535431 23818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:14.541986 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 23751
I20260926 20:39:14.542364 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:14.545168 23731 external_mini_cluster.cc:1428] Setting key 3ea6ab341161b4758fca88f3b14deed1
I20260926 20:39:14.549960 23818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988: Bootstrap starting.
I20260926 20:39:14.554563 23818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:14.556100 23818 log.cc:824] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:14.559854 23818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988: No bootstrap required, opened a new log
I20260926 20:39:14.571795 23818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04af2f7e17b2438fbbe38565c55c4988" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 37893 } }
I20260926 20:39:14.572230 23818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:14.572357 23818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04af2f7e17b2438fbbe38565c55c4988, State: Initialized, Role: FOLLOWER
I20260926 20:39:14.573048 23818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04af2f7e17b2438fbbe38565c55c4988" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 37893 } }
I20260926 20:39:14.573295 23818 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:14.573432 23818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:14.573622 23818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:14.576836 23818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04af2f7e17b2438fbbe38565c55c4988" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 37893 } }
I20260926 20:39:14.577464 23818 leader_election.cc:302] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04af2f7e17b2438fbbe38565c55c4988; no voters:
I20260926 20:39:14.578146 23818 leader_election.cc:288] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:14.578225 23823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:14.579429 23823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [term 1 LEADER]: Becoming Leader. State: Replica: 04af2f7e17b2438fbbe38565c55c4988, State: Running, Role: LEADER
I20260926 20:39:14.579953 23823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04af2f7e17b2438fbbe38565c55c4988" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 37893 } }
I20260926 20:39:14.580606 23818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [sys.catalog]: configured and running, proceeding with master startup.
Sep 26 20:39:14 dist-test-slave-4d5t krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:14.583768 23824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04af2f7e17b2438fbbe38565c55c4988" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04af2f7e17b2438fbbe38565c55c4988" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 37893 } } }
I20260926 20:39:14.583694 23825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04af2f7e17b2438fbbe38565c55c4988. Latest consensus state: current_term: 1 leader_uuid: "04af2f7e17b2438fbbe38565c55c4988" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04af2f7e17b2438fbbe38565c55c4988" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 37893 } } }
I20260926 20:39:14.584316 23824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:14.584318 23825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:14.585458 23829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:14.605152 23820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:14.550268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52584 (local address 127.23.44.254:37893)
0926 20:39:14.551440 (+ 1172us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:14.551458 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:14.552902 (+ 1444us) server_negotiation.cc:407] Connection header received
0926 20:39:14.565967 (+ 13065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:14.565995 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:14.566693 (+ 698us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:14.567363 (+ 670us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:14.571795 (+ 4432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:14.573669 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:14.576242 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:14.577104 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:14.577264 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:14.586812 (+ 9548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:14.586831 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:14.586861 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:14.586944 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:14.594693 (+ 7749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:14.596415 (+ 1722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:14.596430 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:14.596444 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:14.596831 (+ 387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:14.597688 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:14.597696 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:14.597700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:14.598789 (+ 1089us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:14.599247 (+ 458us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:14.603028 (+ 3781us) server_negotiation.cc:299] Negotiation successful
0926 20:39:14.603577 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":543,"threads_started":1}
I20260926 20:39:14.605271 23829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:14.617367 23829 catalog_manager.cc:1347] Generated new cluster ID: 8cce58b90c1a4681af699d3bdee46d48
I20260926 20:39:14.617506 23829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:14.626884 23829 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:14.628878 23829 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:14.635741 23829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04af2f7e17b2438fbbe38565c55c4988: Generated new TSK 0
I20260926 20:39:14.636861 23829 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:39:14.689335 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:37893
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:15.100121 23844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:15.100493 23844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:15.100701 23844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:15.109266 23844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:15.109441 23844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:15.109632 23844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:39:15.122314 23844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:37893
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:15.123850 23844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:15.125401 23844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:15.141144 23849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:15.141206 23852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:15.141175 23850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:15.142311 23844 server_base.cc:1034] running on GCE node
I20260926 20:39:15.143023 23844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:15.143972 23844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:15.145309 23844 hybrid_clock.cc:648] HybridClock initialized: now 1790455155145222 us; error 34 us; skew 500 ppm
I20260926 20:39:15.145850 23844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:15 dist-test-slave-4d5t krb5kdc[23738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455155, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:15.149750 23844 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:39:15.151342 23844 webserver.cc:460] Webserver started at http://127.23.44.193:44079/ using document root <none> and password file <none>
I20260926 20:39:15.152050 23844 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:15.152163 23844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:15.152423 23844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:15.155628 23844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "17fa8353342d414dac77605a9d62a592"
format_stamp: "Formatted at 2026-09-26 20:39:15 on dist-test-slave-4d5t"
server_key: "b93a3419d18610ca27657efdd5003874"
server_key_iv: "08a0b99437a9559f54193c9770856c97"
server_key_version: "encryptionkey@0"
I20260926 20:39:15.156445 23844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "17fa8353342d414dac77605a9d62a592"
format_stamp: "Formatted at 2026-09-26 20:39:15 on dist-test-slave-4d5t"
server_key: "b93a3419d18610ca27657efdd5003874"
server_key_iv: "08a0b99437a9559f54193c9770856c97"
server_key_version: "encryptionkey@0"
I20260926 20:39:15.162421 23844 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.005s
I20260926 20:39:15.166304 23859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:15.168277 23844 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260926 20:39:15.168502 23844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "17fa8353342d414dac77605a9d62a592"
format_stamp: "Formatted at 2026-09-26 20:39:15 on dist-test-slave-4d5t"
server_key: "b93a3419d18610ca27657efdd5003874"
server_key_iv: "08a0b99437a9559f54193c9770856c97"
server_key_version: "encryptionkey@0"
I20260926 20:39:15.168792 23844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:15.184928 23844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:15.189388 23844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:15.189704 23844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:15.190719 23844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:15.192097 23844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:15.192171 23844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:15.192302 23844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:15.192379 23844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:15.232604 23844 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:39205
I20260926 20:39:15.232703 23971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:39205 every 8 connection(s)
I20260926 20:39:15.233949 23844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:39:15.243027 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 23844
I20260926 20:39:15.243263 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:39:15.243860 23731 external_mini_cluster.cc:1428] Setting key 93101e33fbac3ae00d4f54d7ff2a125e
Sep 26 20:39:15 dist-test-slave-4d5t krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455155, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:15.257896 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:15.237057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:40047 (local address 127.23.44.254:37893)
0926 20:39:15.237540 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:15.237544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:15.238612 (+ 1068us) server_negotiation.cc:407] Connection header received
0926 20:39:15.240459 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:15.240465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:15.240600 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:15.240726 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:15.242328 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:15.243230 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:15.245065 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:15.245590 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:15.245729 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:15.250653 (+ 4924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:15.250659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:15.250663 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:15.250700 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:15.253465 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:15.254655 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:15.254660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:15.254663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:15.254816 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:15.255458 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:15.255462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:15.255465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:15.255763 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:15.255892 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:15.257303 (+ 1411us) server_negotiation.cc:299] Negotiation successful
0926 20:39:15.257572 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":362,"thread_start_us":237,"threads_started":1}
I20260926 20:39:15.259366 23975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:15.237684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:37893 (local address 127.23.44.193:40047)
0926 20:39:15.238424 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:15.238464 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:15.240165 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:15.240978 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:15.240987 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:15.241532 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:15.242149 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:15.242167 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:15.243413 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:15.243417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:15.244916 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:15.244929 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:15.245850 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:15.245857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:15.246135 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:15.247385 (+ 1250us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:15.247410 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:15.250352 (+ 2942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:15.253656 (+ 3304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:15.253670 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:15.253695 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:15.254460 (+ 765us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:15.254955 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:15.254964 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:15.254972 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:15.255279 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:15.255915 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:15.255927 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:15.256506 (+ 579us) client_negotiation.cc:769] Sending connection context
0926 20:39:15.256891 (+ 385us) client_negotiation.cc:240] Negotiation successful
0926 20:39:15.257611 (+ 720us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":282,"threads_started":1}
I20260926 20:39:15.261451 23972 heartbeater.cc:347] Connected to a master server at 127.23.44.254:37893
I20260926 20:39:15.261788 23972 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
I20260926 20:39:15.262584 23972 heartbeater.cc:511] Master 127.23.44.254:37893 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
I20260926 20:39:15.265152 23783 ts_manager.cc:194] Registered new tserver with Master: 17fa8353342d414dac77605a9d62a592 (127.23.44.193:39205)
I20260926 20:39:15.266677 23783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:40047
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:39:15.289487 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:37893
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:15.646010 23978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:15.646299 23978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:15.646395 23978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:15.654826 23978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:15.654979 23978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:15.655098 23978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:39:15.665400 23978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:37893
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:15.666688 23978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:15.668032 23978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:15.681996 23983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:15.682072 23984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:15.682283 23986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:15.683174 23978 server_base.cc:1034] running on GCE node
I20260926 20:39:15.683789 23978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:15.684847 23978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:15.686102 23978 hybrid_clock.cc:648] HybridClock initialized: now 1790455155686044 us; error 48 us; skew 500 ppm
I20260926 20:39:15.686452 23978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:15 dist-test-slave-4d5t krb5kdc[23738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455155, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:15.690316 23978 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:39:15.692164 23978 webserver.cc:460] Webserver started at http://127.23.44.194:40775/ using document root <none> and password file <none>
I20260926 20:39:15.692992 23978 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:15.693145 23978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:15.693466 23978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:15.696357 23978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "8038c398f5aa425daf6f2fef1469e721"
format_stamp: "Formatted at 2026-09-26 20:39:15 on dist-test-slave-4d5t"
server_key: "d140683d651742c0b2de641c4f9e15d9"
server_key_iv: "ebb373aea347bc758eccd7566602502e"
server_key_version: "encryptionkey@0"
I20260926 20:39:15.697171 23978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "8038c398f5aa425daf6f2fef1469e721"
format_stamp: "Formatted at 2026-09-26 20:39:15 on dist-test-slave-4d5t"
server_key: "d140683d651742c0b2de641c4f9e15d9"
server_key_iv: "ebb373aea347bc758eccd7566602502e"
server_key_version: "encryptionkey@0"
I20260926 20:39:15.702211 23978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:39:15.705826 23993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:15.707302 23978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:39:15.707490 23978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "8038c398f5aa425daf6f2fef1469e721"
format_stamp: "Formatted at 2026-09-26 20:39:15 on dist-test-slave-4d5t"
server_key: "d140683d651742c0b2de641c4f9e15d9"
server_key_iv: "ebb373aea347bc758eccd7566602502e"
server_key_version: "encryptionkey@0"
I20260926 20:39:15.707698 23978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:15.729760 23978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:15.735317 23978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:15.735719 23978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:15.736802 23978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:15.738260 23978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:15.738348 23978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:15.738447 23978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:15.738497 23978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:15.780143 23978 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:44313
I20260926 20:39:15.780184 24105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:44313 every 8 connection(s)
I20260926 20:39:15.781901 23978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:39:15.791023 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 23978
I20260926 20:39:15.791270 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:39:15.791776 23731 external_mini_cluster.cc:1428] Setting key fb6a42174f3d68ea98f44e3665b43ff3
Sep 26 20:39:15 dist-test-slave-4d5t krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455155, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:15.806341 24108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:15.784931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:60701 (local address 127.23.44.254:37893)
0926 20:39:15.786302 (+ 1371us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:15.786310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:15.787980 (+ 1670us) server_negotiation.cc:407] Connection header received
0926 20:39:15.790278 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:15.790283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:15.790374 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:15.790589 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:15.792622 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:15.793524 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:15.795158 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:15.795594 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:15.795734 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:15.799969 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:15.799974 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:15.799978 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:15.800012 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:15.803026 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:15.803799 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:15.803807 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:15.803812 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:15.803979 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:15.804447 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:15.804454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:15.804457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:15.804785 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:15.804977 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:15.805714 (+ 737us) server_negotiation.cc:299] Negotiation successful
0926 20:39:15.805951 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1203,"thread_start_us":1057,"threads_started":1}
I20260926 20:39:15.807354 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:15.786741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:37893 (local address 127.23.44.194:60701)
0926 20:39:15.787762 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:15.787818 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:15.789929 (+ 2111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:15.790858 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:15.790868 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:15.791593 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:15.792315 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:15.792336 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:15.793703 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:15.793714 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:15.795001 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:15.795014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:15.795838 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:15.795847 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:15.796222 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:15.797122 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:15.797153 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:15.799714 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:15.803177 (+ 3463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:15.803188 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:15.803214 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:15.803661 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:15.804081 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:15.804086 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:15.804090 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:15.804331 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:15.804928 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:15.804937 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:15.805322 (+ 385us) client_negotiation.cc:769] Sending connection context
0926 20:39:15.805634 (+ 312us) client_negotiation.cc:240] Negotiation successful
0926 20:39:15.806189 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":403,"threads_started":1}
I20260926 20:39:15.809047 24106 heartbeater.cc:347] Connected to a master server at 127.23.44.254:37893
I20260926 20:39:15.809444 24106 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
I20260926 20:39:15.810458 24106 heartbeater.cc:511] Master 127.23.44.254:37893 requested a full tablet report, sending...
I20260926 20:39:15.812294 23783 ts_manager.cc:194] Registered new tserver with Master: 8038c398f5aa425daf6f2fef1469e721 (127.23.44.194:44313)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
I20260926 20:39:15.813517 23783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:60701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:39:15.836796 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:37893
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:37245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:16.200062 24112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:16.200403 24112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:16.200568 24112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:16.208974 24112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:16.209103 24112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:16.209224 24112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:39:16.220330 24112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:37245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:37893
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:16.221727 24112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:16.223567 24112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:16.240428 24120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:16.240540 24117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:16.240605 24118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:16.241755 24112 server_base.cc:1034] running on GCE node
I20260926 20:39:16.242444 24112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:16.243474 24112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:16.244674 24112 hybrid_clock.cc:648] HybridClock initialized: now 1790455156244621 us; error 50 us; skew 500 ppm
I20260926 20:39:16.245029 24112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:16 dist-test-slave-4d5t krb5kdc[23738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455156, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:16.248466 24112 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:39:16.250227 24112 webserver.cc:460] Webserver started at http://127.23.44.195:35785/ using document root <none> and password file <none>
I20260926 20:39:16.250941 24112 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:16.251070 24112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:16.251394 24112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:16.254341 24112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "bb96468adaa04e3c8efd971acd6de9eb"
format_stamp: "Formatted at 2026-09-26 20:39:16 on dist-test-slave-4d5t"
server_key: "933a08609f2c34f458af9b927b3dbe8e"
server_key_iv: "2080e91684fb0a2bbbc2148a6c24cc78"
server_key_version: "encryptionkey@0"
I20260926 20:39:16.255134 24112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "bb96468adaa04e3c8efd971acd6de9eb"
format_stamp: "Formatted at 2026-09-26 20:39:16 on dist-test-slave-4d5t"
server_key: "933a08609f2c34f458af9b927b3dbe8e"
server_key_iv: "2080e91684fb0a2bbbc2148a6c24cc78"
server_key_version: "encryptionkey@0"
I20260926 20:39:16.260344 24112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 20:39:16.264389 24127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:16.266484 24112 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:39:16.266798 24112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "bb96468adaa04e3c8efd971acd6de9eb"
format_stamp: "Formatted at 2026-09-26 20:39:16 on dist-test-slave-4d5t"
server_key: "933a08609f2c34f458af9b927b3dbe8e"
server_key_iv: "2080e91684fb0a2bbbc2148a6c24cc78"
server_key_version: "encryptionkey@0"
I20260926 20:39:16.267107 24112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:16.270498 23972 heartbeater.cc:503] Master 127.23.44.254:37893 was elected leader, sending a full tablet report...
I20260926 20:39:16.313351 24112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:16.317946 24112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:16.318218 24112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:16.319144 24112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:16.320503 24112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:16.320653 24112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260926 20:39:16.320751 24112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:16.320804 24112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:16.362648 24239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:46511 every 8 connection(s)
I20260926 20:39:16.362691 24112 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:46511
I20260926 20:39:16.364629 24112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:39:16.368207 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 24112
I20260926 20:39:16.368469 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:39:16.369096 23731 external_mini_cluster.cc:1428] Setting key b910224ab5061ede7285b1b8511794a4
Sep 26 20:39:16 dist-test-slave-4d5t krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455156, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:16.387673 24242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:16.367300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:40129 (local address 127.23.44.254:37893)
0926 20:39:16.367912 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:16.367919 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:16.369156 (+ 1237us) server_negotiation.cc:407] Connection header received
0926 20:39:16.371529 (+ 2373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:16.371535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:16.371647 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:16.371792 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:16.373851 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.374767 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.376599 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.377035 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.377184 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:16.380985 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:16.380991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:16.380994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:16.381028 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:16.384285 (+ 3257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:16.385263 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:16.385269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:16.385273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:16.385404 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:16.385871 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:16.385882 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:16.385885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:16.386153 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:16.386323 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:16.387057 (+ 734us) server_negotiation.cc:299] Negotiation successful
0926 20:39:16.387282 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":281,"threads_started":1}
I20260926 20:39:16.388751 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:16.367983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:37893 (local address 127.23.44.195:40129)
0926 20:39:16.368945 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:16.368991 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:16.371236 (+ 2245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:16.372047 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:16.372056 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:16.372922 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:16.373652 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:16.373673 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.374950 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.374956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:16.376301 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:16.376315 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.377289 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.377296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:16.377567 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:16.378310 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:16.378333 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:16.380733 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:16.384585 (+ 3852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:16.384601 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:16.384623 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:16.385117 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:16.385526 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:16.385531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:16.385535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:16.385736 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:16.386288 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:16.386298 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:16.386683 (+ 385us) client_negotiation.cc:769] Sending connection context
0926 20:39:16.386956 (+ 273us) client_negotiation.cc:240] Negotiation successful
0926 20:39:16.387539 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":455,"threads_started":1}
I20260926 20:39:16.390472 24240 heartbeater.cc:347] Connected to a master server at 127.23.44.254:37893
I20260926 20:39:16.390838 24240 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:16.391646 24240 heartbeater.cc:511] Master 127.23.44.254:37893 requested a full tablet report, sending...
I20260926 20:39:16.393263 23783 ts_manager.cc:194] Registered new tserver with Master: bb96468adaa04e3c8efd971acd6de9eb (127.23.44.195:46511)
I20260926 20:39:16.394066 23783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:40129
I20260926 20:39:16.398679 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 20:39:16 dist-test-slave-4d5t krb5kdc[23738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455156, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 20:39:16 dist-test-slave-4d5t krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455156, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:39:16.449759 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:16.430753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53840 (local address 127.23.44.193:39205)
0926 20:39:16.431638 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:16.431646 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:16.431680 (+ 34us) server_negotiation.cc:407] Connection header received
0926 20:39:16.431745 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:16.431751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:16.431980 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:16.432129 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:16.434242 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.436052 (+ 1810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.437536 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.438171 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.438292 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:16.443096 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:16.443109 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:16.443125 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:16.443168 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:16.446375 (+ 3207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:16.447567 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:16.447582 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:16.447584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:16.447722 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:16.448198 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:16.448202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:16.448204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:16.448612 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:16.448869 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:16.449143 (+ 274us) server_negotiation.cc:299] Negotiation successful
0926 20:39:16.449347 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":476,"threads_started":1}
W20260926 20:39:16.451139 23887 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53840
Sep 26 20:39:16 dist-test-slave-4d5t krb5kdc[23738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 20:39:16 dist-test-slave-4d5t krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:39:16.505398 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:16.488238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.23.44.193:39205)
0926 20:39:16.488488 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:16.488498 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:16.488910 (+ 412us) server_negotiation.cc:407] Connection header received
0926 20:39:16.489157 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:16.489161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:16.489239 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:16.489352 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:16.490855 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.491620 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.493511 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.494015 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.494338 (+ 323us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:16.499115 (+ 4777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:16.499123 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:16.499137 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:16.499178 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:16.502152 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:16.502942 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:16.502946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:16.502948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:16.503070 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:16.503716 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:16.503722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:16.503725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:16.504126 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:16.504470 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:16.504932 (+ 462us) server_negotiation.cc:299] Negotiation successful
0926 20:39:16.505152 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 20:39:16.507108 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 23844
I20260926 20:39:16.521991 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:16.508069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53858 (local address 127.23.44.193:39205)
0926 20:39:16.508268 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:16.508277 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:16.508607 (+ 330us) server_negotiation.cc:407] Connection header received
0926 20:39:16.508804 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:16.508809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:16.508870 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:16.508974 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:16.510134 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.510857 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.512309 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.512823 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.512968 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:16.515208 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:16.515212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:16.515216 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:16.515246 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:16.517744 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:16.518847 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:16.518854 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:16.518857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:16.519053 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:16.519613 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:16.519619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:16.519623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:16.520059 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:16.520415 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:16.521163 (+ 748us) server_negotiation.cc:299] Negotiation successful
0926 20:39:16.521473 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 20:39:16.522774 23967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:16.613313 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 23844
I20260926 20:39:16.630877 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 23978
Sep 26 20:39:16 dist-test-slave-4d5t krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:39:16.651180 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:16.632225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60770 (local address 127.23.44.194:44313)
0926 20:39:16.633088 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:16.633097 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:16.633134 (+ 37us) server_negotiation.cc:407] Connection header received
0926 20:39:16.633222 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:16.633231 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:16.633526 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:16.633702 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:16.635158 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.636517 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.638292 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.638999 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.639129 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:16.644365 (+ 5236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:16.644382 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:16.644402 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:16.644456 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:16.647936 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:16.648842 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:16.648851 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:16.648853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:16.649014 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:16.649464 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:16.649468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:16.649471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:16.649838 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:16.650072 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:16.650652 (+ 580us) server_negotiation.cc:299] Negotiation successful
0926 20:39:16.650891 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":459,"threads_started":1}
I20260926 20:39:16.652640 24101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:16.736184 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 23978
I20260926 20:39:16.759037 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 24112
Sep 26 20:39:16 dist-test-slave-4d5t krb5kdc[23738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:39:16.780328 24257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:16.760292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49532 (local address 127.23.44.195:46511)
0926 20:39:16.761352 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:16.761361 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:16.761398 (+ 37us) server_negotiation.cc:407] Connection header received
0926 20:39:16.761608 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:16.761620 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:16.761965 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:16.762200 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:16.764297 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.765826 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.767455 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:16.768062 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:16.768182 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:16.773617 (+ 5435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:16.773642 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:16.773665 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:16.773730 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:16.777065 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:16.778036 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:16.778043 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:16.778046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:16.778182 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:16.778649 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:16.778654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:16.778656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:16.779067 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:16.779323 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:16.779729 (+ 406us) server_negotiation.cc:299] Negotiation successful
0926 20:39:16.779982 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834,"thread_start_us":490,"threads_started":1}
I20260926 20:39:16.781891 24235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:16.871390 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 24112
I20260926 20:39:16.888281 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 23751
I20260926 20:39:16.889727 23813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:16.997479 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 23751
2026-09-26T20:39:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5341 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:39:17 dist-test-slave-4d5t krb5kdc[24262](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:39:17 dist-test-slave-4d5t krb5kdc[24262](info): set up 2 sockets
Sep 26 20:39:17 dist-test-slave-4d5t krb5kdc[24262](info): commencing operation
krb5kdc: starting...
W20260926 20:39:18.165536 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.006s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:39:18 dist-test-slave-4d5t krb5kdc[24262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:39:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:39:18.360931 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:39149
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:39149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:18.698563 24275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:18.698930 24275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:18.699092 24275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:18.706585 24275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:18.706733 24275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:18.706780 24275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:18.706902 24275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:18.706985 24275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:39:18.717403 24275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:39149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:39149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:18.718848 24275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:18.720290 24275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:18.734117 24280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:18.734156 24281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:18.734920 24283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:18.735721 24275 server_base.cc:1034] running on GCE node
I20260926 20:39:18.736650 24275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:18.738284 24275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:18.739604 24275 hybrid_clock.cc:648] HybridClock initialized: now 1790455158739580 us; error 77 us; skew 500 ppm
I20260926 20:39:18.740020 24275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:18 dist-test-slave-4d5t krb5kdc[24262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455158, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:18.749616 24275 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:39:18.751359 24275 webserver.cc:460] Webserver started at http://127.23.44.254:40645/ using document root <none> and password file <none>
I20260926 20:39:18.752270 24275 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:18.752424 24275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:18.752797 24275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:18.755911 24275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "285fce943657468082aa244bd808a505"
format_stamp: "Formatted at 2026-09-26 20:39:18 on dist-test-slave-4d5t"
server_key: "0e7f90543a7151ea20bca0ad7b465dfb"
server_key_iv: "1f0c974eb3f1c9f749a42f72d338aafb"
server_key_version: "encryptionkey@0"
I20260926 20:39:18.756855 24275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "285fce943657468082aa244bd808a505"
format_stamp: "Formatted at 2026-09-26 20:39:18 on dist-test-slave-4d5t"
server_key: "0e7f90543a7151ea20bca0ad7b465dfb"
server_key_iv: "1f0c974eb3f1c9f749a42f72d338aafb"
server_key_version: "encryptionkey@0"
I20260926 20:39:18.762995 24275 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 20:39:18.767720 24290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:18.769814 24275 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 20:39:18.770078 24275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "285fce943657468082aa244bd808a505"
format_stamp: "Formatted at 2026-09-26 20:39:18 on dist-test-slave-4d5t"
server_key: "0e7f90543a7151ea20bca0ad7b465dfb"
server_key_iv: "1f0c974eb3f1c9f749a42f72d338aafb"
server_key_version: "encryptionkey@0"
I20260926 20:39:18.770363 24275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:18.789608 24275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:18.795554 24275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:18.796020 24275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:18.819612 24275 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:39149
I20260926 20:39:18.819613 24341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:39149 every 8 connection(s)
I20260926 20:39:18.821169 24275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:18.825335 24342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:18.831755 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 24275
I20260926 20:39:18.832052 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:18.832994 23731 external_mini_cluster.cc:1428] Setting key 2455ba7e105b7bc00a968a87516c77d1
I20260926 20:39:18.841800 24342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505: Bootstrap starting.
I20260926 20:39:18.846035 24342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:18.847239 24342 log.cc:824] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505: Log is configured to *not* fsync() on all Append() calls
Sep 26 20:39:18 dist-test-slave-4d5t krb5kdc[24262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:18.851061 24342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505: No bootstrap required, opened a new log
I20260926 20:39:18.862001 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:18.835695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47298 (local address 127.23.44.254:39149)
0926 20:39:18.837430 (+ 1735us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:18.837446 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:18.837502 (+ 56us) server_negotiation.cc:407] Connection header received
0926 20:39:18.839737 (+ 2235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:18.839770 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:18.840310 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:18.840866 (+ 556us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:18.842963 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:18.844756 (+ 1793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:18.846285 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:18.847063 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:18.847296 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:18.852381 (+ 5085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:18.852390 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:18.852409 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:18.852450 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:18.855949 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:18.857285 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:18.857293 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:18.857301 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:18.857565 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:18.858115 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:18.858119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:18.858122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:18.858777 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:18.859052 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:18.859874 (+ 822us) server_negotiation.cc:299] Negotiation successful
0926 20:39:18.860507 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1442,"thread_start_us":891,"threads_started":1}
I20260926 20:39:18.861802 24342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285fce943657468082aa244bd808a505" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39149 } }
I20260926 20:39:18.862604 24342 raft_consensus.cc:374] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:18.862771 24342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285fce943657468082aa244bd808a505, State: Initialized, Role: FOLLOWER
I20260926 20:39:18.863572 24342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285fce943657468082aa244bd808a505" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39149 } }
I20260926 20:39:18.863865 24342 raft_consensus.cc:388] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:18.864096 24342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:18.864365 24342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:18.867585 24342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285fce943657468082aa244bd808a505" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39149 } }
I20260926 20:39:18.868172 24342 leader_election.cc:302] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 285fce943657468082aa244bd808a505; no voters:
I20260926 20:39:18.868891 24342 leader_election.cc:288] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:18.869029 24347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:18.869822 24347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [term 1 LEADER]: Becoming Leader. State: Replica: 285fce943657468082aa244bd808a505, State: Running, Role: LEADER
I20260926 20:39:18.870396 24347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285fce943657468082aa244bd808a505" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39149 } }
I20260926 20:39:18.871204 24342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:39:18.873816 24349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 285fce943657468082aa244bd808a505. Latest consensus state: current_term: 1 leader_uuid: "285fce943657468082aa244bd808a505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285fce943657468082aa244bd808a505" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39149 } } }
I20260926 20:39:18.873771 24348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "285fce943657468082aa244bd808a505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285fce943657468082aa244bd808a505" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39149 } } }
I20260926 20:39:18.874329 24349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:18.874337 24348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:18.877202 24354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:18.882486 24354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:18.893162 24354 catalog_manager.cc:1347] Generated new cluster ID: fb55e5db9fb54d3bbe0afe620ef55d88
I20260926 20:39:18.893323 24354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:18.902822 24354 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:18.904865 24354 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:18.913353 24354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 285fce943657468082aa244bd808a505: Generated new TSK 0
I20260926 20:39:18.914491 24354 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:39:18.964655 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39149
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:19.332170 24368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:19.332479 24368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:19.332597 24368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:19.341061 24368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:19.341197 24368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:19.341403 24368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:39:19.353734 24368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39149
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:19.355620 24368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:19.357682 24368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:19.374152 24376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:19.374498 24373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:19.374207 24374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:19.375154 24368 server_base.cc:1034] running on GCE node
I20260926 20:39:19.376034 24368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:19.377384 24368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:19.378660 24368 hybrid_clock.cc:648] HybridClock initialized: now 1790455159378586 us; error 73 us; skew 500 ppm
I20260926 20:39:19.379019 24368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:19 dist-test-slave-4d5t krb5kdc[24262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455159, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:19.383437 24368 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:39:19.385687 24368 webserver.cc:460] Webserver started at http://127.23.44.193:34493/ using document root <none> and password file <none>
I20260926 20:39:19.386637 24368 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:19.386809 24368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:19.387195 24368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:19.390944 24368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "ade7df506165414086f15cb10407232e"
format_stamp: "Formatted at 2026-09-26 20:39:19 on dist-test-slave-4d5t"
server_key: "02761936a54abb1305ca89d99179711d"
server_key_iv: "00b92b3a988ceaea8220b7d9b608f030"
server_key_version: "encryptionkey@0"
I20260926 20:39:19.391887 24368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "ade7df506165414086f15cb10407232e"
format_stamp: "Formatted at 2026-09-26 20:39:19 on dist-test-slave-4d5t"
server_key: "02761936a54abb1305ca89d99179711d"
server_key_iv: "00b92b3a988ceaea8220b7d9b608f030"
server_key_version: "encryptionkey@0"
I20260926 20:39:19.398499 24368 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260926 20:39:19.402801 24383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:19.404263 24368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:39:19.404464 24368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "ade7df506165414086f15cb10407232e"
format_stamp: "Formatted at 2026-09-26 20:39:19 on dist-test-slave-4d5t"
server_key: "02761936a54abb1305ca89d99179711d"
server_key_iv: "00b92b3a988ceaea8220b7d9b608f030"
server_key_version: "encryptionkey@0"
I20260926 20:39:19.404752 24368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:19.423832 24368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:19.429102 24368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:19.429574 24368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:19.430949 24368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:19.432390 24368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:19.432500 24368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:19.432727 24368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:19.432806 24368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:19.474009 24368 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:35681
I20260926 20:39:19.474056 24495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:35681 every 8 connection(s)
I20260926 20:39:19.475534 24368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:39:19.476918 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 24368
I20260926 20:39:19.477222 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:39:19.478039 23731 external_mini_cluster.cc:1428] Setting key 285c331c8f6091392fe0a3f3bb535b37
Sep 26 20:39:19 dist-test-slave-4d5t krb5kdc[24262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455159, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
I20260926 20:39:19.500478 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:19.478715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:35883 (local address 127.23.44.254:39149)
0926 20:39:19.479317 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:19.479327 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:19.480735 (+ 1408us) server_negotiation.cc:407] Connection header received
0926 20:39:19.482831 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:19.482842 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:19.482948 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:19.483112 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:19.485847 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:19.486686 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:19.488477 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:19.488992 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:19.489144 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:19.493718 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:19.493724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:19.493728 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:19.493766 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:19.496692 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:19.497681 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:19.497687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:19.497690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:19.497902 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:19.498569 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:19.498574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:19.498577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:19.498897 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:19.499022 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:19.500041 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0926 20:39:19.500212 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":304,"threads_started":1}
I20260926 20:39:19.502063 24499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:19.479411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39149 (local address 127.23.44.193:35883)
0926 20:39:19.480527 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:19.480609 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:19.482527 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:19.483712 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:19.483727 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:19.484522 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:19.485576 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:19.485616 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:19.486888 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:19.486893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:19.488283 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:19.488296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:19.489257 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:19.489268 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:19.489549 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:19.490338 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:19.490372 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:19.493352 (+ 2980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:19.496911 (+ 3559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:19.496919 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:19.496934 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:19.497365 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:19.498035 (+ 670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:19.498038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:19.498040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:19.498346 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:19.499048 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:19.499059 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:19.499571 (+ 512us) client_negotiation.cc:769] Sending connection context
0926 20:39:19.499889 (+ 318us) client_negotiation.cc:240] Negotiation successful
0926 20:39:19.500497 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":616,"threads_started":1}
I20260926 20:39:19.503875 24496 heartbeater.cc:347] Connected to a master server at 127.23.44.254:39149
I20260926 20:39:19.504200 24496 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:19.504999 24496 heartbeater.cc:511] Master 127.23.44.254:39149 requested a full tablet report, sending...
I20260926 20:39:19.507683 24307 ts_manager.cc:194] Registered new tserver with Master: ade7df506165414086f15cb10407232e (127.23.44.193:35681)
I20260926 20:39:19.509471 24307 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:35883
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:39:19.524338 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39149
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:19.874132 24502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:19.874591 24502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:19.874771 24502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:19.883026 24502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:19.883198 24502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:19.883401 24502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:39:19.895166 24502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39149
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:19.896858 24502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:19.898484 24502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:19.913450 24510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:19.913936 24507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:19.913709 24508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:19.914973 24502 server_base.cc:1034] running on GCE node
I20260926 20:39:19.915656 24502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:19.916930 24502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:19.918186 24502 hybrid_clock.cc:648] HybridClock initialized: now 1790455159918125 us; error 58 us; skew 500 ppm
I20260926 20:39:19.918555 24502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:19 dist-test-slave-4d5t krb5kdc[24262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455159, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:19.922151 24502 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:39:19.924121 24502 webserver.cc:460] Webserver started at http://127.23.44.194:37259/ using document root <none> and password file <none>
I20260926 20:39:19.924913 24502 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:19.925025 24502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:19.925294 24502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:19.928234 24502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "ddda293643634719baa746d705b9ed7a"
format_stamp: "Formatted at 2026-09-26 20:39:19 on dist-test-slave-4d5t"
server_key: "26562f3047b0a6894fbb7c79037e9c4c"
server_key_iv: "34c4d0e8bd88db9e34c7c18f45ca8e84"
server_key_version: "encryptionkey@0"
I20260926 20:39:19.929066 24502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "ddda293643634719baa746d705b9ed7a"
format_stamp: "Formatted at 2026-09-26 20:39:19 on dist-test-slave-4d5t"
server_key: "26562f3047b0a6894fbb7c79037e9c4c"
server_key_iv: "34c4d0e8bd88db9e34c7c18f45ca8e84"
server_key_version: "encryptionkey@0"
I20260926 20:39:19.934310 24502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:39:19.938369 24517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:19.940151 24502 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:39:19.940322 24502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "ddda293643634719baa746d705b9ed7a"
format_stamp: "Formatted at 2026-09-26 20:39:19 on dist-test-slave-4d5t"
server_key: "26562f3047b0a6894fbb7c79037e9c4c"
server_key_iv: "34c4d0e8bd88db9e34c7c18f45ca8e84"
server_key_version: "encryptionkey@0"
I20260926 20:39:19.940502 24502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:19.957336 24502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:19.961912 24502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:19.962175 24502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:19.963230 24502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:19.964809 24502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:19.964896 24502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:19.965073 24502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:19.965132 24502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:20.002147 24502 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:38087
I20260926 20:39:20.002159 24629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:38087 every 8 connection(s)
I20260926 20:39:20.003775 24502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:39:20.005998 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 24502
I20260926 20:39:20.006194 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:39:20.006639 23731 external_mini_cluster.cc:1428] Setting key 0c7c051a6d9a8ca3659156532954b666
Sep 26 20:39:20 dist-test-slave-4d5t krb5kdc[24262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455159, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
I20260926 20:39:20.028833 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:20.007262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:38293 (local address 127.23.44.254:39149)
0926 20:39:20.007780 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:20.007785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:20.009279 (+ 1494us) server_negotiation.cc:407] Connection header received
0926 20:39:20.011238 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:20.011246 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:20.011337 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:20.011463 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:20.013080 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.013795 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.015393 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.016189 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.016333 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:20.019961 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:20.019971 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:20.019979 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:20.020028 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:20.023861 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:20.025321 (+ 1460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:20.025326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:20.025328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:20.025490 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:20.026243 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:20.026249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:20.026252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:20.026809 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:20.027059 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:20.027912 (+ 853us) server_negotiation.cc:299] Negotiation successful
0926 20:39:20.028180 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":252,"threads_started":1}
I20260926 20:39:20.029887 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:20.008134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39149 (local address 127.23.44.194:38293)
0926 20:39:20.009093 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:20.009147 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:20.010963 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:20.011667 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:20.011675 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:20.012180 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:20.012888 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:20.012918 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.013954 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.013959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:20.015247 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:20.015258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.016437 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.016443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:20.016733 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:20.017439 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:20.017462 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:20.019682 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:20.024133 (+ 4451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:20.024145 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:20.024166 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:20.025114 (+ 948us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:20.025674 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:20.025681 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:20.025686 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:20.026035 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:20.027064 (+ 1029us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:20.027071 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:20.027478 (+ 407us) client_negotiation.cc:769] Sending connection context
0926 20:39:20.027794 (+ 316us) client_negotiation.cc:240] Negotiation successful
0926 20:39:20.028377 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":408,"threads_started":1}
I20260926 20:39:20.031617 24630 heartbeater.cc:347] Connected to a master server at 127.23.44.254:39149
I20260926 20:39:20.031970 24630 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:20.032905 24630 heartbeater.cc:511] Master 127.23.44.254:39149 requested a full tablet report, sending...
I20260926 20:39:20.034647 24307 ts_manager.cc:194] Registered new tserver with Master: ddda293643634719baa746d705b9ed7a (127.23.44.194:38087)
I20260926 20:39:20.035948 24307 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:38293
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:39:20.052263 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39149
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:20.420346 24636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:20.420789 24636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:20.421063 24636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:20.429668 24636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:20.429850 24636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:20.430127 24636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:39:20.443248 24636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39149
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:20.444895 24636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:20.446964 24636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:20.462961 24642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:20.463735 24644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:20.463968 24641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:20.464439 24636 server_base.cc:1034] running on GCE node
I20260926 20:39:20.465178 24636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:20.466231 24636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:20.467501 24636 hybrid_clock.cc:648] HybridClock initialized: now 1790455160467445 us; error 55 us; skew 500 ppm
I20260926 20:39:20.467859 24636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:20 dist-test-slave-4d5t krb5kdc[24262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455160, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:20.471177 24636 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:39:20.474746 24636 webserver.cc:460] Webserver started at http://127.23.44.195:41121/ using document root <none> and password file <none>
I20260926 20:39:20.475579 24636 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:20.475755 24636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:20.476130 24636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:20.479708 24636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "d0cb18fd0cf348ebbee337104663c74b"
format_stamp: "Formatted at 2026-09-26 20:39:20 on dist-test-slave-4d5t"
server_key: "ea7af7087d317576057ad2b5374fdf79"
server_key_iv: "a255c15f49c1c5e0d9d0c12ae58d3203"
server_key_version: "encryptionkey@0"
I20260926 20:39:20.480641 24636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "d0cb18fd0cf348ebbee337104663c74b"
format_stamp: "Formatted at 2026-09-26 20:39:20 on dist-test-slave-4d5t"
server_key: "ea7af7087d317576057ad2b5374fdf79"
server_key_iv: "a255c15f49c1c5e0d9d0c12ae58d3203"
server_key_version: "encryptionkey@0"
I20260926 20:39:20.486297 24636 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:39:20.490260 24651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:20.492067 24636 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:39:20.492311 24636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "d0cb18fd0cf348ebbee337104663c74b"
format_stamp: "Formatted at 2026-09-26 20:39:20 on dist-test-slave-4d5t"
server_key: "ea7af7087d317576057ad2b5374fdf79"
server_key_iv: "a255c15f49c1c5e0d9d0c12ae58d3203"
server_key_version: "encryptionkey@0"
I20260926 20:39:20.492537 24636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:20.508037 24636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:20.513442 24636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:20.513546 24496 heartbeater.cc:503] Master 127.23.44.254:39149 was elected leader, sending a full tablet report...
I20260926 20:39:20.513801 24636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:20.514971 24636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:20.516413 24636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:20.516517 24636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:20.516718 24636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:20.516798 24636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:20.553597 24636 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:33455
I20260926 20:39:20.553678 24763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:33455 every 8 connection(s)
I20260926 20:39:20.554925 24636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:39:20.563305 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 24636
I20260926 20:39:20.563539 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:39:20.564002 23731 external_mini_cluster.cc:1428] Setting key c050dd22571b5f5c2f50f89f1d65f553
Sep 26 20:39:20 dist-test-slave-4d5t krb5kdc[24262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455160, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:20.580214 24766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:20.558367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:33391 (local address 127.23.44.254:39149)
0926 20:39:20.558871 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:20.558878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:20.560102 (+ 1224us) server_negotiation.cc:407] Connection header received
0926 20:39:20.562516 (+ 2414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:20.562521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:20.562599 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:20.562723 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:20.564615 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.565781 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.567517 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.568201 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.568346 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:20.572371 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:20.572376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:20.572380 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:20.572415 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:20.575806 (+ 3391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:20.577057 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:20.577068 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:20.577071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:20.577216 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:20.578093 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:20.578099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:20.578103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:20.578463 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:20.578661 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:20.579464 (+ 803us) server_negotiation.cc:299] Negotiation successful
0926 20:39:20.579747 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":214,"threads_started":1}
I20260926 20:39:20.581391 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:20.559095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39149 (local address 127.23.44.195:33391)
0926 20:39:20.559910 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:20.559952 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:20.562074 (+ 2122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:20.562952 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:20.562962 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:20.563671 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:20.564390 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:20.564412 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.565951 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.565962 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:20.567348 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:20.567361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.568457 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.568465 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:20.568789 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:20.569532 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:20.569559 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:20.572113 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:20.576100 (+ 3987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:20.576144 (+ 44us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:20.576175 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:20.576898 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:20.577559 (+ 661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:20.577569 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:20.577575 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:20.577926 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:20.578632 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:20.578644 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:20.579046 (+ 402us) client_negotiation.cc:769] Sending connection context
0926 20:39:20.579348 (+ 302us) client_negotiation.cc:240] Negotiation successful
0926 20:39:20.579944 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":340,"threads_started":1}
I20260926 20:39:20.583078 24764 heartbeater.cc:347] Connected to a master server at 127.23.44.254:39149
I20260926 20:39:20.583539 24764 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:20.584360 24764 heartbeater.cc:511] Master 127.23.44.254:39149 requested a full tablet report, sending...
I20260926 20:39:20.586206 24307 ts_manager.cc:194] Registered new tserver with Master: d0cb18fd0cf348ebbee337104663c74b (127.23.44.195:33455)
I20260926 20:39:20.587054 24307 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:33391
I20260926 20:39:20.593470 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:39:20.614288 24766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:20.600498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47304 (local address 127.23.44.254:39149)
0926 20:39:20.600740 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:20.600744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:20.601669 (+ 925us) server_negotiation.cc:407] Connection header received
0926 20:39:20.601834 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:20.601838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:20.601926 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:20.602007 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:20.603477 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.604194 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.605454 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.605856 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.605959 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:20.608383 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:20.608387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:20.608390 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:20.608423 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:20.611066 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:20.611871 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:20.611877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:20.611880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:20.612120 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:20.612714 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:20.612718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:20.612720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:20.613291 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:20.613533 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:20.613856 (+ 323us) server_negotiation.cc:299] Negotiation successful
0926 20:39:20.614045 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260926 20:39:20.621055 24307 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:39:20.628499 24307 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:39:20.658166 24777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:20.641494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:33455 (local address 127.0.0.1:60680)
0926 20:39:20.642304 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:20.642343 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:20.642537 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:20.643533 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:20.643543 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:20.643574 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:20.643887 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:20.643921 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.650353 (+ 6432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.650363 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:20.653820 (+ 3457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:20.653846 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.657118 (+ 3272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.657127 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:20.657356 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:20.657421 (+ 65us) client_negotiation.cc:769] Sending connection context
0926 20:39:20.657582 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 20:39:20.657809 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":414,"threads_started":1}
I20260926 20:39:20.658396 24778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:20.641572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60680 (local address 127.23.44.195:33455)
0926 20:39:20.642852 (+ 1280us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:20.642861 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:20.642894 (+ 33us) server_negotiation.cc:407] Connection header received
0926 20:39:20.643022 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:20.643031 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:20.643304 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:20.644188 (+ 884us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:20.646497 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.648721 (+ 2224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.654816 (+ 6095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.656882 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.657030 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:20.657275 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:20.657647 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 20:39:20.657843 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1087,"thread_start_us":934,"threads_started":1}
I20260926 20:39:20.663640 24780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:20.643159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59402 (local address 127.23.44.194:38087)
0926 20:39:20.654680 (+ 11521us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:20.654691 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:20.654730 (+ 39us) server_negotiation.cc:407] Connection header received
0926 20:39:20.654841 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:20.654852 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:20.655159 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:20.655407 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:20.656292 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.658144 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.661459 (+ 3315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.662580 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.662747 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:20.662950 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:20.663175 (+ 225us) server_negotiation.cc:299] Negotiation successful
0926 20:39:20.663347 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11353,"thread_start_us":11244,"threads_started":1}
I20260926 20:39:20.663699 24779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:20.642844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:38087 (local address 127.0.0.1:59402)
0926 20:39:20.643378 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:20.643399 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:20.643502 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:20.655646 (+ 12144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:20.655652 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:20.655682 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:20.656083 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:20.656096 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.658332 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.658338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:20.661265 (+ 2927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:20.661286 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.662812 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.662819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:20.662988 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:20.663023 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 20:39:20.663135 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 20:39:20.663274 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":85,"thread_start_us":282,"threads_started":1}
I20260926 20:39:20.663293 24699 tablet_service.cc:1467] Processing CreateTablet for tablet d6eedfa29d0842cb9e3a934d0c2bcdf2 (DEFAULT_TABLE table=test-table [id=5b8d9e03ce1f4e7c8deab44584b23184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:20.664849 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:20.644220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38468 (local address 127.23.44.193:35681)
0926 20:39:20.656000 (+ 11780us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:20.656010 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:20.656039 (+ 29us) server_negotiation.cc:407] Connection header received
0926 20:39:20.656137 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:20.656147 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:20.656401 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:20.657302 (+ 901us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:20.657540 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.659040 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.661897 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.663392 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.663551 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:20.663798 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:20.664393 (+ 595us) server_negotiation.cc:299] Negotiation successful
0926 20:39:20.664521 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11581,"thread_start_us":11439,"threads_started":1}
I20260926 20:39:20.664924 24781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:20.643564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:35681 (local address 127.0.0.1:38468)
0926 20:39:20.653602 (+ 10038us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:20.653636 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:20.653834 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:20.656653 (+ 2819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:20.656658 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:20.656685 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:20.657076 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:20.657090 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.659190 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.659198 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:20.661695 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:20.661709 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:20.663963 (+ 2254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:20.663970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:20.664157 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:20.664197 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 20:39:20.664314 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 20:39:20.664457 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9815,"mutex_wait_us":1,"thread_start_us":10400,"threads_started":1}
I20260926 20:39:20.665141 24699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6eedfa29d0842cb9e3a934d0c2bcdf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:20.668284 24565 tablet_service.cc:1467] Processing CreateTablet for tablet d6eedfa29d0842cb9e3a934d0c2bcdf2 (DEFAULT_TABLE table=test-table [id=5b8d9e03ce1f4e7c8deab44584b23184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:20.669605 24565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6eedfa29d0842cb9e3a934d0c2bcdf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:20.671579 24431 tablet_service.cc:1467] Processing CreateTablet for tablet d6eedfa29d0842cb9e3a934d0c2bcdf2 (DEFAULT_TABLE table=test-table [id=5b8d9e03ce1f4e7c8deab44584b23184]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:20.673204 24431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6eedfa29d0842cb9e3a934d0c2bcdf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:20.680694 24783 tablet_bootstrap.cc:492] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b: Bootstrap starting.
I20260926 20:39:20.683560 24784 tablet_bootstrap.cc:492] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a: Bootstrap starting.
I20260926 20:39:20.686805 24783 tablet_bootstrap.cc:654] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:20.687520 24784 tablet_bootstrap.cc:654] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:20.688004 24783 log.cc:824] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:20.688882 24784 log.cc:824] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:20.691036 24786 tablet_bootstrap.cc:492] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e: Bootstrap starting.
I20260926 20:39:20.691690 24783 tablet_bootstrap.cc:492] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b: No bootstrap required, opened a new log
I20260926 20:39:20.691987 24783 ts_tablet_manager.cc:1397] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260926 20:39:20.693370 24784 tablet_bootstrap.cc:492] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a: No bootstrap required, opened a new log
I20260926 20:39:20.693663 24784 ts_tablet_manager.cc:1397] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 20:39:20.695112 24786 tablet_bootstrap.cc:654] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:20.696805 24786 log.cc:824] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:20.700950 24786 tablet_bootstrap.cc:492] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e: No bootstrap required, opened a new log
I20260926 20:39:20.701280 24786 ts_tablet_manager.cc:1397] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260926 20:39:20.701267 24783 raft_consensus.cc:348] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cb18fd0cf348ebbee337104663c74b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33455 } } peers { permanent_uuid: "ddda293643634719baa746d705b9ed7a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38087 } } peers { permanent_uuid: "ade7df506165414086f15cb10407232e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35681 } }
I20260926 20:39:20.701709 24783 raft_consensus.cc:374] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:20.701817 24783 raft_consensus.cc:729] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0cb18fd0cf348ebbee337104663c74b, State: Initialized, Role: FOLLOWER
I20260926 20:39:20.702474 24783 consensus_queue.cc:260] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cb18fd0cf348ebbee337104663c74b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33455 } } peers { permanent_uuid: "ddda293643634719baa746d705b9ed7a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38087 } } peers { permanent_uuid: "ade7df506165414086f15cb10407232e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35681 } }
I20260926 20:39:20.703635 24764 heartbeater.cc:503] Master 127.23.44.254:39149 was elected leader, sending a full tablet report...
I20260926 20:39:20.704339 24783 ts_tablet_manager.cc:1428] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260926 20:39:20.706200 24784 raft_consensus.cc:348] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cb18fd0cf348ebbee337104663c74b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33455 } } peers { permanent_uuid: "ddda293643634719baa746d705b9ed7a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38087 } } peers { permanent_uuid: "ade7df506165414086f15cb10407232e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35681 } }
I20260926 20:39:20.706692 24784 raft_consensus.cc:374] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:20.706794 24784 raft_consensus.cc:729] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddda293643634719baa746d705b9ed7a, State: Initialized, Role: FOLLOWER
I20260926 20:39:20.707566 24784 consensus_queue.cc:260] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cb18fd0cf348ebbee337104663c74b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33455 } } peers { permanent_uuid: "ddda293643634719baa746d705b9ed7a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38087 } } peers { permanent_uuid: "ade7df506165414086f15cb10407232e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35681 } }
I20260926 20:39:20.709903 24630 heartbeater.cc:503] Master 127.23.44.254:39149 was elected leader, sending a full tablet report...
I20260926 20:39:20.710764 24784 ts_tablet_manager.cc:1428] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a: Time spent starting tablet: real 0.017s user 0.010s sys 0.005s
I20260926 20:39:20.714082 24786 raft_consensus.cc:348] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cb18fd0cf348ebbee337104663c74b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33455 } } peers { permanent_uuid: "ddda293643634719baa746d705b9ed7a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38087 } } peers { permanent_uuid: "ade7df506165414086f15cb10407232e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35681 } }
I20260926 20:39:20.714560 24786 raft_consensus.cc:374] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:20.714673 24786 raft_consensus.cc:729] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade7df506165414086f15cb10407232e, State: Initialized, Role: FOLLOWER
I20260926 20:39:20.715449 24786 consensus_queue.cc:260] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cb18fd0cf348ebbee337104663c74b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33455 } } peers { permanent_uuid: "ddda293643634719baa746d705b9ed7a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38087 } } peers { permanent_uuid: "ade7df506165414086f15cb10407232e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35681 } }
I20260926 20:39:20.717011 24786 ts_tablet_manager.cc:1428] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260926 20:39:20.728654 24497 tablet.cc:2369] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:39:20.756381 24631 tablet.cc:2369] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:39:20.806633 24765 tablet.cc:2369] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:39:21.043226 24789 raft_consensus.cc:482] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:39:21.043661 24789 raft_consensus.cc:504] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cb18fd0cf348ebbee337104663c74b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33455 } } peers { permanent_uuid: "ddda293643634719baa746d705b9ed7a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38087 } } peers { permanent_uuid: "ade7df506165414086f15cb10407232e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35681 } }
I20260926 20:39:21.046082 24789 leader_election.cc:288] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddda293643634719baa746d705b9ed7a (127.23.44.194:38087), ade7df506165414086f15cb10407232e (127.23.44.193:35681)
I20260926 20:39:21.054826 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:21.046503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:38087 (local address 127.23.44.195:57885)
0926 20:39:21.046712 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:21.046734 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:21.046866 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:21.047557 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:21.047562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:21.047593 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:21.047818 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:21.047830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.049413 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.049429 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:21.051852 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:21.051868 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.053902 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.053911 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:21.054153 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:21.054215 (+ 62us) client_negotiation.cc:769] Sending connection context
0926 20:39:21.054345 (+ 130us) client_negotiation.cc:240] Negotiation successful
0926 20:39:21.054500 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 20:39:21.054951 24780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:21.046762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:57885 (local address 127.23.44.194:38087)
0926 20:39:21.047050 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:21.047059 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:21.047090 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:39:21.047198 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:21.047206 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:21.047325 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:21.047520 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:21.048011 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.049231 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.052153 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.053679 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.053886 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:21.053972 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:21.054398 (+ 426us) server_negotiation.cc:299] Negotiation successful
0926 20:39:21.054578 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260926 20:39:21.056267 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:21.047330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:39689 (local address 127.23.44.193:35681)
0926 20:39:21.047553 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:21.047559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:21.047827 (+ 268us) server_negotiation.cc:407] Connection header received
0926 20:39:21.048026 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:21.048029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:21.048091 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:21.048166 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:21.048913 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.049878 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.053618 (+ 3740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.055124 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.055274 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:21.055359 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:21.055974 (+ 615us) server_negotiation.cc:299] Negotiation successful
0926 20:39:21.056092 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 20:39:21.056192 24585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6eedfa29d0842cb9e3a934d0c2bcdf2" candidate_uuid: "d0cb18fd0cf348ebbee337104663c74b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddda293643634719baa746d705b9ed7a" is_pre_election: true
I20260926 20:39:21.056849 24585 raft_consensus.cc:2454] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0cb18fd0cf348ebbee337104663c74b in term 0.
I20260926 20:39:21.056968 24792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:21.046977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:35681 (local address 127.23.44.195:39689)
0926 20:39:21.047684 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:21.047711 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:21.047851 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:21.048219 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:21.048225 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:21.048252 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:21.048717 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:21.048731 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.050166 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.050173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:21.052757 (+ 2584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:21.052776 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.055536 (+ 2760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.055541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:21.055755 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:21.055784 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 20:39:21.056467 (+ 683us) client_negotiation.cc:240] Negotiation successful
0926 20:39:21.056636 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":317,"threads_started":1}
I20260926 20:39:21.057984 24655 leader_election.cc:302] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0cb18fd0cf348ebbee337104663c74b, ddda293643634719baa746d705b9ed7a; no voters:
I20260926 20:39:21.058171 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6eedfa29d0842cb9e3a934d0c2bcdf2" candidate_uuid: "d0cb18fd0cf348ebbee337104663c74b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade7df506165414086f15cb10407232e" is_pre_election: true
I20260926 20:39:21.058491 24789 raft_consensus.cc:2781] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:39:21.058632 24789 raft_consensus.cc:482] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:39:21.058729 24789 raft_consensus.cc:3037] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:21.058756 24451 raft_consensus.cc:2454] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0cb18fd0cf348ebbee337104663c74b in term 0.
I20260926 20:39:21.061465 24789 raft_consensus.cc:504] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cb18fd0cf348ebbee337104663c74b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33455 } } peers { permanent_uuid: "ddda293643634719baa746d705b9ed7a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38087 } } peers { permanent_uuid: "ade7df506165414086f15cb10407232e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35681 } }
I20260926 20:39:21.062094 24789 leader_election.cc:288] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [CANDIDATE]: Term 1 election: Requested vote from peers ddda293643634719baa746d705b9ed7a (127.23.44.194:38087), ade7df506165414086f15cb10407232e (127.23.44.193:35681)
I20260926 20:39:21.062693 24585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6eedfa29d0842cb9e3a934d0c2bcdf2" candidate_uuid: "d0cb18fd0cf348ebbee337104663c74b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddda293643634719baa746d705b9ed7a"
I20260926 20:39:21.062811 24451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6eedfa29d0842cb9e3a934d0c2bcdf2" candidate_uuid: "d0cb18fd0cf348ebbee337104663c74b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade7df506165414086f15cb10407232e"
I20260926 20:39:21.062911 24585 raft_consensus.cc:3037] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:21.063088 24451 raft_consensus.cc:3037] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:21.065285 24585 raft_consensus.cc:2454] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ddda293643634719baa746d705b9ed7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0cb18fd0cf348ebbee337104663c74b in term 1.
I20260926 20:39:21.065565 24451 raft_consensus.cc:2454] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P ade7df506165414086f15cb10407232e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0cb18fd0cf348ebbee337104663c74b in term 1.
I20260926 20:39:21.065728 24655 leader_election.cc:302] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0cb18fd0cf348ebbee337104663c74b, ddda293643634719baa746d705b9ed7a; no voters:
I20260926 20:39:21.066251 24789 raft_consensus.cc:2781] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:21.066923 24789 raft_consensus.cc:686] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [term 1 LEADER]: Becoming Leader. State: Replica: d0cb18fd0cf348ebbee337104663c74b, State: Running, Role: LEADER
I20260926 20:39:21.067582 24789 consensus_queue.cc:237] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cb18fd0cf348ebbee337104663c74b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33455 } } peers { permanent_uuid: "ddda293643634719baa746d705b9ed7a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38087 } } peers { permanent_uuid: "ade7df506165414086f15cb10407232e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35681 } }
I20260926 20:39:21.073319 24306 catalog_manager.cc:5495] T d6eedfa29d0842cb9e3a934d0c2bcdf2 P d0cb18fd0cf348ebbee337104663c74b reported cstate change: term changed from 0 to 1, leader changed from <none> to d0cb18fd0cf348ebbee337104663c74b (127.23.44.195). New cstate: current_term: 1 leader_uuid: "d0cb18fd0cf348ebbee337104663c74b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cb18fd0cf348ebbee337104663c74b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddda293643634719baa746d705b9ed7a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ade7df506165414086f15cb10407232e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35681 } health_report { overall_health: UNKNOWN } } }
Sep 26 20:39:21 dist-test-slave-4d5t krb5kdc[24262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:39:21.122263 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:21.104626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38480 (local address 127.23.44.193:35681)
0926 20:39:21.104819 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:21.104825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:21.105332 (+ 507us) server_negotiation.cc:407] Connection header received
0926 20:39:21.105552 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:21.105557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:21.105632 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:21.105749 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:21.107317 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.108056 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.109405 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.109981 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.110124 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:21.115342 (+ 5218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:21.115365 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:21.115386 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:21.115440 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:21.118870 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:21.119794 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:21.119802 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:21.119805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:21.119959 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:21.120412 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:21.120422 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:21.120425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:21.120887 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:21.121219 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:21.121565 (+ 346us) server_negotiation.cc:299] Negotiation successful
0926 20:39:21.121818 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
Sep 26 20:39:21 dist-test-slave-4d5t krb5kdc[24262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455161, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 20:39:21 dist-test-slave-4d5t krb5kdc[24262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455161, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:39:21.176123 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:21.159502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38490 (local address 127.23.44.193:35681)
0926 20:39:21.159852 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:21.159861 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:21.160256 (+ 395us) server_negotiation.cc:407] Connection header received
0926 20:39:21.160506 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:21.160513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:21.160664 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:21.160788 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:21.162255 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.163068 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.164461 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.165004 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.165152 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:21.170092 (+ 4940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:21.170098 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:21.170101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:21.170138 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:21.173430 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:21.174133 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:21.174140 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:21.174143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:21.174279 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:21.174677 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:21.174683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:21.174687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:21.174963 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:21.175277 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:21.175527 (+ 250us) server_negotiation.cc:299] Negotiation successful
0926 20:39:21.175754 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 20:39:21 dist-test-slave-4d5t krb5kdc[24262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 20:39:21 dist-test-slave-4d5t krb5kdc[24262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:39:21.227133 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:21.210765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38492 (local address 127.23.44.193:35681)
0926 20:39:21.211025 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:21.211034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:21.211337 (+ 303us) server_negotiation.cc:407] Connection header received
0926 20:39:21.211555 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:21.211560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:21.211623 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:21.211729 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:21.213374 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.214120 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.215867 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.216351 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.216501 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:21.221535 (+ 5034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:21.221542 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:21.221546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:21.221587 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:21.224492 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:21.225333 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:21.225337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:21.225340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:21.225478 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:21.225895 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:21.225898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:21.225900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:21.226176 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:21.226373 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:21.226651 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 20:39:21.226848 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260926 20:39:21.229065 24411 tablet_service.cc:2908] Rejecting scan request for tablet d6eedfa29d0842cb9e3a934d0c2bcdf2: Uninitialized: clean time has not yet been initialized
I20260926 20:39:21.232143 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 24368
I20260926 20:39:21.249131 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:21.233613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38502 (local address 127.23.44.193:35681)
0926 20:39:21.233841 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:21.233848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:21.234226 (+ 378us) server_negotiation.cc:407] Connection header received
0926 20:39:21.234348 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:21.234352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:21.234429 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:21.234528 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:21.235879 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.236782 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.238234 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.238678 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.238802 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:21.242150 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:21.242156 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:21.242160 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:21.242203 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:21.246083 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:21.246960 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:21.246964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:21.246966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:21.247111 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:21.247746 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:21.247750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:21.247752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:21.248055 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:21.248299 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:21.248726 (+ 427us) server_negotiation.cc:299] Negotiation successful
0926 20:39:21.248890 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:39:21.249837 24491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:21.340328 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 24368
I20260926 20:39:21.359320 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 24502
Sep 26 20:39:21 dist-test-slave-4d5t krb5kdc[24262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:39:21.377224 24780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:21.360226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59414 (local address 127.23.44.194:38087)
0926 20:39:21.360418 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:21.360423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:21.360445 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:39:21.360743 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:21.360748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:21.360822 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:21.360932 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:21.362634 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.363374 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.364923 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.365335 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.365475 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:21.369862 (+ 4387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:21.369884 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:21.369907 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:21.369959 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:21.373516 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:21.374322 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:21.374329 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:21.374332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:21.374461 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:21.374917 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:21.374921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:21.374924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:21.375349 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:21.375593 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:21.376428 (+ 835us) server_negotiation.cc:299] Negotiation successful
0926 20:39:21.376789 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 20:39:21.378087 24625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:21.476001 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 24502
I20260926 20:39:21.497224 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 24636
Sep 26 20:39:21 dist-test-slave-4d5t krb5kdc[24262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:39:21.514288 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:21.497965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60682 (local address 127.23.44.195:33455)
0926 20:39:21.498490 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:21.498496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:21.498515 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:39:21.498570 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:21.498574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:21.498635 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:21.498748 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:21.500007 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.500967 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.502328 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:21.502737 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:21.502861 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:21.507279 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:21.507294 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:21.507311 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:21.507361 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:21.510962 (+ 3601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:21.511740 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:21.511753 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:21.511755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:21.511923 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:21.512438 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:21.512445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:21.512448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:21.513218 (+ 770us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:21.513489 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:21.513747 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 20:39:21.513939 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":314,"threads_started":1}
I20260926 20:39:21.515072 24759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:21.615123 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 24636
I20260926 20:39:21.631878 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 24275
I20260926 20:39:21.633131 24337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:21.744041 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 24275
2026-09-26T20:39:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4747 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:39:21 dist-test-slave-4d5t krb5kdc[24813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:39:21 dist-test-slave-4d5t krb5kdc[24813](info): set up 2 sockets
Sep 26 20:39:21 dist-test-slave-4d5t krb5kdc[24813](info): commencing operation
krb5kdc: starting...
W20260926 20:39:22.880885 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:39:23 dist-test-slave-4d5t krb5kdc[24813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:39:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:39:23.088735 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33483
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:33483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:23.424152 24826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:23.424515 24826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:23.424700 24826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:23.432192 24826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:23.432364 24826 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 20:39:23.432459 24826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:23.432618 24826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:23.432696 24826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:23.432791 24826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:39:23.443131 24826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:33483
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:23.444536 24826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:23.445885 24826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:23.459175 24831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:23.459115 24832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:23.459096 24834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:23.459884 24826 server_base.cc:1034] running on GCE node
I20260926 20:39:23.460906 24826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:23.462543 24826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:23.464003 24826 hybrid_clock.cc:648] HybridClock initialized: now 1790455163463869 us; error 43 us; skew 500 ppm
I20260926 20:39:23.464468 24826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:23 dist-test-slave-4d5t krb5kdc[24813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455163, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:23.468250 24826 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:39:23.469975 24826 webserver.cc:460] Webserver started at http://127.23.44.254:41283/ using document root <none> and password file <none>
I20260926 20:39:23.470726 24826 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:23.470822 24826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:23.471105 24826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:23.474884 24826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "91816fa7f5674802837097dd41ab1782"
format_stamp: "Formatted at 2026-09-26 20:39:23 on dist-test-slave-4d5t"
server_key: "77aa35377a5b5d69020f954a90ea6966"
server_key_iv: "4a46e78348dcafffcdc0fbe09395e96e"
server_key_version: "encryptionkey@0"
I20260926 20:39:23.475966 24826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "91816fa7f5674802837097dd41ab1782"
format_stamp: "Formatted at 2026-09-26 20:39:23 on dist-test-slave-4d5t"
server_key: "77aa35377a5b5d69020f954a90ea6966"
server_key_iv: "4a46e78348dcafffcdc0fbe09395e96e"
server_key_version: "encryptionkey@0"
I20260926 20:39:23.482180 24826 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260926 20:39:23.486387 24841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:23.488323 24826 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:39:23.488648 24826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "91816fa7f5674802837097dd41ab1782"
format_stamp: "Formatted at 2026-09-26 20:39:23 on dist-test-slave-4d5t"
server_key: "77aa35377a5b5d69020f954a90ea6966"
server_key_iv: "4a46e78348dcafffcdc0fbe09395e96e"
server_key_version: "encryptionkey@0"
I20260926 20:39:23.488927 24826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:23.510360 24826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:23.515909 24826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:23.516247 24826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:23.540524 24826 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:33483
I20260926 20:39:23.540542 24902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:33483 every 8 connection(s)
I20260926 20:39:23.542019 24826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:23.545760 24903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:23.549165 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 24826
I20260926 20:39:23.549431 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:23.550194 23731 external_mini_cluster.cc:1428] Setting key 5d801f1d507177432825bf60bac0434c
I20260926 20:39:23.558964 24903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782: Bootstrap starting.
I20260926 20:39:23.563009 24903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:23.564409 24903 log.cc:824] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782: Log is configured to *not* fsync() on all Append() calls
Sep 26 20:39:23 dist-test-slave-4d5t krb5kdc[24813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:23.567771 24903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782: No bootstrap required, opened a new log
I20260926 20:39:23.575634 24906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:23.552285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33094 (local address 127.23.44.254:33483)
0926 20:39:23.553063 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:23.553074 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:23.553115 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:39:23.554751 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:23.554772 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:23.555167 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:23.555514 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:23.557353 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:23.558829 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:23.560606 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:23.561347 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:23.561522 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:23.567034 (+ 5512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:23.567045 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:23.567069 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:23.567127 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:23.570804 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:23.571809 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:23.571818 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:23.571825 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:23.572075 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:23.572642 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:23.572647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:23.572649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:23.573300 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:23.573558 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:23.574038 (+ 480us) server_negotiation.cc:299] Negotiation successful
0926 20:39:23.574555 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":379,"threads_started":1}
I20260926 20:39:23.577463 24903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91816fa7f5674802837097dd41ab1782" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33483 } }
I20260926 20:39:23.577858 24903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:23.577991 24903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91816fa7f5674802837097dd41ab1782, State: Initialized, Role: FOLLOWER
I20260926 20:39:23.578596 24903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91816fa7f5674802837097dd41ab1782" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33483 } }
I20260926 20:39:23.578910 24903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:23.579048 24903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:23.579218 24903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:23.581733 24903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91816fa7f5674802837097dd41ab1782" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33483 } }
I20260926 20:39:23.582279 24903 leader_election.cc:302] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91816fa7f5674802837097dd41ab1782; no voters:
I20260926 20:39:23.582885 24903 leader_election.cc:288] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:23.583081 24908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:23.583894 24908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [term 1 LEADER]: Becoming Leader. State: Replica: 91816fa7f5674802837097dd41ab1782, State: Running, Role: LEADER
I20260926 20:39:23.584862 24903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:39:23.584856 24908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91816fa7f5674802837097dd41ab1782" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33483 } }
I20260926 20:39:23.587843 24909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91816fa7f5674802837097dd41ab1782" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91816fa7f5674802837097dd41ab1782" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33483 } } }
I20260926 20:39:23.588229 24909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:23.588688 24910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91816fa7f5674802837097dd41ab1782. Latest consensus state: current_term: 1 leader_uuid: "91816fa7f5674802837097dd41ab1782" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91816fa7f5674802837097dd41ab1782" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33483 } } }
I20260926 20:39:23.588967 24910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:23.590917 24917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:23.596094 24917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:23.606660 24917 catalog_manager.cc:1347] Generated new cluster ID: dd4612beebad47bbb8c3d2f83bc862e6
I20260926 20:39:23.606796 24917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:23.629639 24917 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:23.631750 24917 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:23.642078 24917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782: Generated new TSK 0
I20260926 20:39:23.642987 24917 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:39:23.700817 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33483
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:24.077363 24929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:24.077706 24929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:24.077883 24929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:24.085477 24929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:24.085644 24929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:24.085807 24929 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:39:24.085964 24929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:39:24.096789 24929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.44.254:33483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:24.098165 24929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:24.099552 24929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:24.113569 24935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:24.113701 24934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:24.113617 24937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:24.114598 24929 server_base.cc:1034] running on GCE node
I20260926 20:39:24.115248 24929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:24.117041 24929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:24.118311 24929 hybrid_clock.cc:648] HybridClock initialized: now 1790455164118282 us; error 89 us; skew 500 ppm
I20260926 20:39:24.118752 24929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:24 dist-test-slave-4d5t krb5kdc[24813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455164, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:24.128036 24929 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:39:24.130519 24929 webserver.cc:460] Webserver started at http://127.23.44.193:39083/ using document root <none> and password file <none>
I20260926 20:39:24.131531 24929 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:24.131655 24929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:24.132225 24929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:24.135664 24929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "984c547b182647b99c7818bd53fec93e"
format_stamp: "Formatted at 2026-09-26 20:39:24 on dist-test-slave-4d5t"
server_key: "74d3442aaab04f43d3e6b76f8edc775c"
server_key_iv: "4bb71761477f1863bfcd0d7897eaa163"
server_key_version: "encryptionkey@0"
I20260926 20:39:24.136490 24929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "984c547b182647b99c7818bd53fec93e"
format_stamp: "Formatted at 2026-09-26 20:39:24 on dist-test-slave-4d5t"
server_key: "74d3442aaab04f43d3e6b76f8edc775c"
server_key_iv: "4bb71761477f1863bfcd0d7897eaa163"
server_key_version: "encryptionkey@0"
I20260926 20:39:24.143652 24929 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260926 20:39:24.148988 24944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:24.150720 24929 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260926 20:39:24.150943 24929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "984c547b182647b99c7818bd53fec93e"
format_stamp: "Formatted at 2026-09-26 20:39:24 on dist-test-slave-4d5t"
server_key: "74d3442aaab04f43d3e6b76f8edc775c"
server_key_iv: "4bb71761477f1863bfcd0d7897eaa163"
server_key_version: "encryptionkey@0"
I20260926 20:39:24.151214 24929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:24.174099 24929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:24.178956 24929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:24.179265 24929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:24.182214 24929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:24.182338 24929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:24.182502 24929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:24.182585 24929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:39:24 dist-test-slave-4d5t krb5kdc[24813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455164, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:24.206441 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:24.182718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:35633 (local address 127.23.44.254:33483)
0926 20:39:24.183520 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:24.183524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:24.184511 (+ 987us) server_negotiation.cc:407] Connection header received
0926 20:39:24.186776 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:24.186782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:24.186874 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:24.186995 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:24.189290 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.190444 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.193305 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.193945 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.194089 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:24.198799 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:24.198809 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:24.198815 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:24.198874 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:24.202488 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:24.203523 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:24.203529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:24.203533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:24.203706 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:24.204356 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:24.204362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:24.204365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:24.204749 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:24.204873 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:24.206006 (+ 1133us) server_negotiation.cc:299] Negotiation successful
0926 20:39:24.206194 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":605,"threads_started":1}
I20260926 20:39:24.208635 24954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:24.183481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33483 (local address 127.23.44.193:35633)
0926 20:39:24.184344 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:24.184383 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:24.186475 (+ 2092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:24.187254 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:24.187269 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:24.188022 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:24.189111 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:24.189142 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.190605 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.190610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:24.193133 (+ 2523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:24.193148 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.194232 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.194239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:24.194525 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:24.195251 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:24.195285 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:24.198439 (+ 3154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:24.202661 (+ 4222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:24.202672 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:24.202692 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:24.203376 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:24.203850 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:24.203858 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:24.203864 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:24.204191 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:24.204895 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:24.204909 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:24.205512 (+ 603us) client_negotiation.cc:769] Sending connection context
0926 20:39:24.205908 (+ 396us) client_negotiation.cc:240] Negotiation successful
0926 20:39:24.206637 (+ 729us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":405,"threads_started":1}
I20260926 20:39:24.242285 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:24.228246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:46041 (local address 127.23.44.254:33483)
0926 20:39:24.228483 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:24.228490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:24.228758 (+ 268us) server_negotiation.cc:407] Connection header received
0926 20:39:24.228992 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:24.228999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:24.229079 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:24.229211 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:24.230250 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.231033 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.232523 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.233144 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.233308 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:24.236251 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:24.236256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:24.236260 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:24.236294 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:24.239131 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:24.239923 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:24.239931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:24.239940 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:24.240068 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:24.240474 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:24.240482 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:24.240486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:24.240856 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:24.241034 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:24.241573 (+ 539us) server_negotiation.cc:299] Negotiation successful
0926 20:39:24.241835 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 20:39:24.242816 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:24.227704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33483 (local address 127.23.44.193:46041)
0926 20:39:24.228617 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:24.228641 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:24.228848 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:24.229247 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:24.229251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:24.229776 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:24.230088 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:24.230098 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.231197 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.231204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:24.232362 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:24.232375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.233411 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.233422 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:24.233679 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:24.234575 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:24.234596 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:24.236005 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:24.239302 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:24.239308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:24.239313 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:24.239778 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:24.240201 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:24.240208 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:24.240212 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:24.240355 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:24.240987 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:24.240992 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:24.241272 (+ 280us) client_negotiation.cc:769] Sending connection context
0926 20:39:24.241471 (+ 199us) client_negotiation.cc:240] Negotiation successful
0926 20:39:24.241885 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":499,"threads_started":1}
I20260926 20:39:24.244243 24929 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:36953
I20260926 20:39:24.244244 25064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:36953 every 8 connection(s)
I20260926 20:39:24.245645 24929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:39:24.248075 25065 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33483
I20260926 20:39:24.248344 25065 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:24.249106 25065 heartbeater.cc:511] Master 127.23.44.254:33483 requested a full tablet report, sending...
I20260926 20:39:24.251242 24858 ts_manager.cc:194] Registered new tserver with Master: 984c547b182647b99c7818bd53fec93e (127.23.44.193:36953)
I20260926 20:39:24.252897 24858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:35633
I20260926 20:39:24.255914 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 24929
I20260926 20:39:24.256131 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:39:24.256815 23731 external_mini_cluster.cc:1428] Setting key 5ef96e00809a6569f9cc9d45a4f65d76
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:39:24.300724 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33483
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:24.640167 25069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:24.640697 25069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:24.640904 25069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:24.649906 25069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:24.650085 25069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:24.650255 25069 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:39:24.650404 25069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:39:24.661108 25069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.44.254:33483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:24.662485 25069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:24.663805 25069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:24.677326 25075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:24.677376 25077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:24.677352 25074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:24.678395 25069 server_base.cc:1034] running on GCE node
I20260926 20:39:24.679036 25069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:24.680028 25069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:24.681296 25069 hybrid_clock.cc:648] HybridClock initialized: now 1790455164681234 us; error 39 us; skew 500 ppm
I20260926 20:39:24.681635 25069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:24 dist-test-slave-4d5t krb5kdc[24813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455164, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:24.685483 25069 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:39:24.687163 25069 webserver.cc:460] Webserver started at http://127.23.44.194:46183/ using document root <none> and password file <none>
I20260926 20:39:24.687953 25069 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:24.688088 25069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:24.688417 25069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:24.691481 25069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "be72ed9e76f3461b80a3fe51cdabedc7"
format_stamp: "Formatted at 2026-09-26 20:39:24 on dist-test-slave-4d5t"
server_key: "582b4c5e37b277244ed27668cf6bf5a5"
server_key_iv: "eeee9144e4169f67a4938df96bb6837c"
server_key_version: "encryptionkey@0"
I20260926 20:39:24.692306 25069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "be72ed9e76f3461b80a3fe51cdabedc7"
format_stamp: "Formatted at 2026-09-26 20:39:24 on dist-test-slave-4d5t"
server_key: "582b4c5e37b277244ed27668cf6bf5a5"
server_key_iv: "eeee9144e4169f67a4938df96bb6837c"
server_key_version: "encryptionkey@0"
I20260926 20:39:24.697932 25069 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 20:39:24.701965 25084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:24.703421 25069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 20:39:24.703612 25069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "be72ed9e76f3461b80a3fe51cdabedc7"
format_stamp: "Formatted at 2026-09-26 20:39:24 on dist-test-slave-4d5t"
server_key: "582b4c5e37b277244ed27668cf6bf5a5"
server_key_iv: "eeee9144e4169f67a4938df96bb6837c"
server_key_version: "encryptionkey@0"
I20260926 20:39:24.703861 25069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:24.721786 25069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:24.727123 25069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:24.727499 25069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:24.731254 25069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:24.731386 25069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:24.731530 25069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:24.731594 25069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:39:24 dist-test-slave-4d5t krb5kdc[24813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455164, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:24.759951 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:24.732214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33483 (local address 127.23.44.194:55609)
0926 20:39:24.733322 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:24.733371 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:24.735266 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:24.736097 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:24.736109 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:24.736948 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:24.738018 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:24.738051 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.739513 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.739521 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:24.741686 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:24.741701 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.742557 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.742564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:24.742808 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:24.743666 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:24.743706 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:24.748233 (+ 4527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:24.753851 (+ 5618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:24.753877 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:24.753907 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:24.754647 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:24.755212 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:24.755220 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:24.755232 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:24.755718 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:24.756688 (+ 970us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:24.756700 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:24.757207 (+ 507us) client_negotiation.cc:769] Sending connection context
0926 20:39:24.757621 (+ 414us) client_negotiation.cc:240] Negotiation successful
0926 20:39:24.758312 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":494,"threads_started":1}
I20260926 20:39:24.761123 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:24.731590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:55609 (local address 127.23.44.254:33483)
0926 20:39:24.731803 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:24.731810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:24.733529 (+ 1719us) server_negotiation.cc:407] Connection header received
0926 20:39:24.735554 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:24.735562 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:24.735676 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:24.735815 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:24.738242 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.739336 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.741871 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.742307 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.742452 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:24.748639 (+ 6187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:24.748645 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:24.748665 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:24.748722 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:24.753576 (+ 4854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:24.754870 (+ 1294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:24.754874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:24.754876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:24.755056 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:24.755919 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:24.755926 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:24.755932 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:24.756434 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:24.756669 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:24.760617 (+ 3948us) server_negotiation.cc:299] Negotiation successful
0926 20:39:24.760878 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:39:24.784895 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:24.766793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33483 (local address 127.23.44.194:53857)
0926 20:39:24.768099 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:24.768139 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:24.768353 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:24.768799 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:24.768804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:24.769605 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:24.770132 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:24.770148 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.772156 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.772161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:24.774027 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:24.774050 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.775329 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.775334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:24.775449 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:24.776127 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:24.776144 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:24.777094 (+ 950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:24.781662 (+ 4568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:24.781672 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:24.781678 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:24.782376 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:24.782881 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:24.782884 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:24.782887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:24.783005 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:24.783856 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:24.783859 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:24.784020 (+ 161us) client_negotiation.cc:769] Sending connection context
0926 20:39:24.784144 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 20:39:24.784391 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1078,"thread_start_us":872,"threads_started":1}
I20260926 20:39:24.785104 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:24.766721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:53857 (local address 127.23.44.254:33483)
0926 20:39:24.767028 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:24.767037 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:24.768259 (+ 1222us) server_negotiation.cc:407] Connection header received
0926 20:39:24.768492 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:24.768495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:24.768622 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:24.768766 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:24.770351 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.771844 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.774238 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:24.775113 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:24.775298 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:24.777384 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:24.777394 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:24.777402 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:24.777452 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:24.781437 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:24.782528 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:24.782537 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:24.782543 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:24.782743 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:24.783116 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:24.783123 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:24.783130 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:24.783709 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:24.783899 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:24.784300 (+ 401us) server_negotiation.cc:299] Negotiation successful
0926 20:39:24.784624 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260926 20:39:24.793958 25069 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:39963
I20260926 20:39:24.794023 25203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:39963 every 8 connection(s)
I20260926 20:39:24.795255 25069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:39:24.797780 25204 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33483
I20260926 20:39:24.798045 25204 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:24.798743 25204 heartbeater.cc:511] Master 127.23.44.254:33483 requested a full tablet report, sending...
I20260926 20:39:24.800889 24858 ts_manager.cc:194] Registered new tserver with Master: be72ed9e76f3461b80a3fe51cdabedc7 (127.23.44.194:39963)
I20260926 20:39:24.801728 24858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:55609
I20260926 20:39:24.802529 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 25069
I20260926 20:39:24.802747 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:39:24.803177 23731 external_mini_cluster.cc:1428] Setting key 720166741d985d0e64f85c42e541df8f
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:39:24.848649 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33483
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:25.200456 25208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:25.200817 25208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:25.200973 25208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:25.209208 25208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:25.209327 25208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:25.209432 25208 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:39:25.209517 25208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:39:25.219576 25208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.44.254:33483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:25.221431 25208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:25.222934 25208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:25.239857 25213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:25.239876 25214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:25.239853 25216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:25.241274 25208 server_base.cc:1034] running on GCE node
I20260926 20:39:25.242019 25208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:25.243093 25208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:25.244362 25208 hybrid_clock.cc:648] HybridClock initialized: now 1790455165244277 us; error 57 us; skew 500 ppm
I20260926 20:39:25.244747 25208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:25 dist-test-slave-4d5t krb5kdc[24813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455165, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:25.247967 25208 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:39:25.249671 25208 webserver.cc:460] Webserver started at http://127.23.44.195:42627/ using document root <none> and password file <none>
I20260926 20:39:25.250362 25208 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:25.250451 25208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:25.250701 25208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:25.253698 25208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "44d6886f29a2411bbbd4f4b94d139a08"
format_stamp: "Formatted at 2026-09-26 20:39:25 on dist-test-slave-4d5t"
server_key: "d3c5c7f000d758ca4f6a885c60528e77"
server_key_iv: "47a4815ff19777602f5f7dcbdb06805d"
server_key_version: "encryptionkey@0"
I20260926 20:39:25.254498 25208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "44d6886f29a2411bbbd4f4b94d139a08"
format_stamp: "Formatted at 2026-09-26 20:39:25 on dist-test-slave-4d5t"
server_key: "d3c5c7f000d758ca4f6a885c60528e77"
server_key_iv: "47a4815ff19777602f5f7dcbdb06805d"
server_key_version: "encryptionkey@0"
I20260926 20:39:25.256035 25065 heartbeater.cc:503] Master 127.23.44.254:33483 was elected leader, sending a full tablet report...
I20260926 20:39:25.260468 25208 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 20:39:25.264772 25223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:25.266417 25208 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260926 20:39:25.266613 25208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "44d6886f29a2411bbbd4f4b94d139a08"
format_stamp: "Formatted at 2026-09-26 20:39:25 on dist-test-slave-4d5t"
server_key: "d3c5c7f000d758ca4f6a885c60528e77"
server_key_iv: "47a4815ff19777602f5f7dcbdb06805d"
server_key_version: "encryptionkey@0"
I20260926 20:39:25.266871 25208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:25.279821 25208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:25.284899 25208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:25.285234 25208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:25.288312 25208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:25.288434 25208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:25.288599 25208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:25.288708 25208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Sep 26 20:39:25 dist-test-slave-4d5t krb5kdc[24813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455165, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:25.316268 25232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.288914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:52101 (local address 127.23.44.254:33483)
0926 20:39:25.291955 (+ 3041us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:25.291963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:25.291990 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:39:25.293866 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:25.293875 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:25.294105 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:25.294241 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:25.296572 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.298056 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.299858 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.300785 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.300925 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.306483 (+ 5558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:25.306488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:25.306491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:25.306531 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:25.311554 (+ 5023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:25.312510 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:25.312518 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:25.312521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:25.312721 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:25.313346 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:25.313352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:25.313355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:25.313792 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:25.313955 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:25.315618 (+ 1663us) server_negotiation.cc:299] Negotiation successful
0926 20:39:25.315866 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2906,"thread_start_us":2795,"threads_started":1}
I20260926 20:39:25.317454 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.289660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33483 (local address 127.23.44.195:52101)
0926 20:39:25.291503 (+ 1843us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:25.291572 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:25.293592 (+ 2020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:25.294524 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:25.294534 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:25.295384 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:25.296265 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.296287 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.298237 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.298241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.299710 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.299723 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.301072 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.301085 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.301362 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.302338 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:25.302385 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:25.306204 (+ 3819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:25.311832 (+ 5628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:25.311845 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:25.311865 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:25.312369 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:25.312870 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:25.312877 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:25.312882 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:25.313192 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:25.313955 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:25.313966 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:25.314896 (+ 930us) client_negotiation.cc:769] Sending connection context
0926 20:39:25.315524 (+ 628us) client_negotiation.cc:240] Negotiation successful
0926 20:39:25.316132 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1426,"thread_start_us":553,"threads_started":1}
I20260926 20:39:25.340030 25232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.323447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:39889 (local address 127.23.44.254:33483)
0926 20:39:25.323654 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:25.323659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:25.324635 (+ 976us) server_negotiation.cc:407] Connection header received
0926 20:39:25.324905 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:25.324911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:25.325002 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:25.325121 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:25.326119 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.326893 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.328353 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.329088 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.329279 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.332869 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:25.332880 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:25.332885 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:25.332932 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:25.336667 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:25.337801 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:25.337808 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:25.337812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:25.337965 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:25.338380 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:25.338387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:25.338391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:25.338700 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:25.338880 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:25.339299 (+ 419us) server_negotiation.cc:299] Negotiation successful
0926 20:39:25.339592 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 20:39:25.340538 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.323465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33483 (local address 127.23.44.195:39889)
0926 20:39:25.324460 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:25.324486 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:25.324735 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:25.325153 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:25.325158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:25.325654 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:25.325963 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.325972 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.327046 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.327052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.328196 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.328208 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.329534 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.329539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.329754 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.330753 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:25.330790 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:25.332511 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:25.337169 (+ 4658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:25.337176 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:25.337180 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:25.337645 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:25.338101 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:25.338116 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:25.338120 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:25.338259 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:25.338828 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:25.338835 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:25.339034 (+ 199us) client_negotiation.cc:769] Sending connection context
0926 20:39:25.339206 (+ 172us) client_negotiation.cc:240] Negotiation successful
0926 20:39:25.339603 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":581,"threads_started":1}
I20260926 20:39:25.360059 25208 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:45423
I20260926 20:39:25.360098 25343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:45423 every 8 connection(s)
I20260926 20:39:25.361519 25208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:39:25.364284 25344 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33483
I20260926 20:39:25.364490 25344 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:25.365212 25344 heartbeater.cc:511] Master 127.23.44.254:33483 requested a full tablet report, sending...
I20260926 20:39:25.366916 24858 ts_manager.cc:194] Registered new tserver with Master: 44d6886f29a2411bbbd4f4b94d139a08 (127.23.44.195:45423)
I20260926 20:39:25.368027 24858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:52101
I20260926 20:39:25.370323 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 25208
I20260926 20:39:25.370522 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:39:25.370985 23731 external_mini_cluster.cc:1428] Setting key f9efedda2afd72e06540a2764a78a45d
I20260926 20:39:25.374536 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 20:39:25 dist-test-slave-4d5t krb5kdc[24813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455165, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 20:39:25 dist-test-slave-4d5t krb5kdc[24813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455165, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:25.426246 25232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.409766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55890 (local address 127.23.44.254:33483)
0926 20:39:25.410034 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:25.410044 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:25.410367 (+ 323us) server_negotiation.cc:407] Connection header received
0926 20:39:25.410539 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:25.410544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:25.410657 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:25.410776 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:25.412813 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.413527 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.414779 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.415193 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.415293 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.420218 (+ 4925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:25.420223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:25.420227 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:25.420264 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:25.423596 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:25.424369 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:25.424373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:25.424375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:25.424502 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:25.424990 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:25.424993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:25.424996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:25.425274 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:25.425483 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:25.425761 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 20:39:25.425914 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260926 20:39:25.431058 24858 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:39:25.438614 24858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:39:25.463590 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.452344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45558 (local address 127.23.44.193:36953)
0926 20:39:25.454413 (+ 2069us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:25.454420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:25.455073 (+ 653us) server_negotiation.cc:407] Connection header received
0926 20:39:25.455365 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:25.455375 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:25.455589 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:25.455763 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:25.456924 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.458325 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.460767 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.462052 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.462219 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.462456 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:25.463072 (+ 616us) server_negotiation.cc:299] Negotiation successful
0926 20:39:25.463263 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1927,"thread_start_us":1790,"threads_started":1}
I20260926 20:39:25.464211 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.451716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:36953 (local address 127.0.0.1:45558)
0926 20:39:25.454960 (+ 3244us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:25.454978 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:25.455167 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:25.456031 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:25.456036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:25.456060 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:25.456760 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.456774 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.458517 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.458523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.460612 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.460631 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.462257 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.462264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.462792 (+ 528us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.462852 (+ 60us) client_negotiation.cc:769] Sending connection context
0926 20:39:25.463686 (+ 834us) client_negotiation.cc:240] Negotiation successful
0926 20:39:25.463800 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3109,"mutex_wait_us":6,"thread_start_us":987,"threads_started":1}
I20260926 20:39:25.464401 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.455184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:39963 (local address 127.0.0.1:51870)
0926 20:39:25.456040 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:25.456056 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:25.456963 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:25.457568 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:25.457574 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:25.457605 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:25.457928 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.457939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.459702 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.459708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.461699 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.461709 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.463527 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.463533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.463728 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.463759 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 20:39:25.463846 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 20:39:25.464001 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":3,"spinlock_wait_cycles":1833728,"thread_start_us":637,"threads_started":1}
I20260926 20:39:25.464416 25359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.451728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43152 (local address 127.23.44.195:45423)
0926 20:39:25.454205 (+ 2477us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:25.454216 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:25.454250 (+ 34us) server_negotiation.cc:407] Connection header received
0926 20:39:25.454362 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:25.454370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:25.455297 (+ 927us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:25.455639 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:25.457099 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.458677 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.461333 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.462829 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.462954 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.463165 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:25.463931 (+ 766us) server_negotiation.cc:299] Negotiation successful
0926 20:39:25.464091 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2288,"thread_start_us":2099,"threads_started":1}
I20260926 20:39:25.464861 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.451387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:45423 (local address 127.0.0.1:43152)
0926 20:39:25.453198 (+ 1811us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:25.453259 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:25.453476 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:25.455963 (+ 2487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:25.455974 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:25.456016 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:25.456843 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.456854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.458825 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.458830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.461158 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.461172 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.463304 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.463312 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.463481 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.463516 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 20:39:25.463592 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 20:39:25.463812 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1497,"mutex_wait_us":80,"spinlock_wait_cycles":218496,"thread_start_us":3254,"threads_started":1}
I20260926 20:39:25.464952 25362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.455651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51870 (local address 127.23.44.194:39963)
0926 20:39:25.456494 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:25.456502 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:25.456537 (+ 35us) server_negotiation.cc:407] Connection header received
0926 20:39:25.457156 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:25.457167 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:25.457375 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:25.457566 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:25.458131 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.459553 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.461870 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.463086 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.463211 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.463388 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:25.464508 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0926 20:39:25.464707 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":546,"threads_started":1}
I20260926 20:39:25.468433 24995 tablet_service.cc:1467] Processing CreateTablet for tablet 90a739def14b457895d5885e9504e3df (DEFAULT_TABLE table=test-table [id=40d69176f7f64aa0b5285d36233af762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:25.468433 25279 tablet_service.cc:1467] Processing CreateTablet for tablet 90a739def14b457895d5885e9504e3df (DEFAULT_TABLE table=test-table [id=40d69176f7f64aa0b5285d36233af762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:25.469898 24995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90a739def14b457895d5885e9504e3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:25.469898 25279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90a739def14b457895d5885e9504e3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:25.470541 25134 tablet_service.cc:1467] Processing CreateTablet for tablet 90a739def14b457895d5885e9504e3df (DEFAULT_TABLE table=test-table [id=40d69176f7f64aa0b5285d36233af762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:25.471947 25134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90a739def14b457895d5885e9504e3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:25.484265 25363 tablet_bootstrap.cc:492] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7: Bootstrap starting.
I20260926 20:39:25.486496 25364 tablet_bootstrap.cc:492] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e: Bootstrap starting.
I20260926 20:39:25.488310 25365 tablet_bootstrap.cc:492] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08: Bootstrap starting.
I20260926 20:39:25.489100 25363 tablet_bootstrap.cc:654] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:25.490454 25364 tablet_bootstrap.cc:654] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:25.490586 25363 log.cc:824] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:25.491696 25365 tablet_bootstrap.cc:654] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:25.492002 25364 log.cc:824] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:25.492820 25365 log.cc:824] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:25.496347 25363 tablet_bootstrap.cc:492] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7: No bootstrap required, opened a new log
I20260926 20:39:25.496426 25364 tablet_bootstrap.cc:492] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e: No bootstrap required, opened a new log
I20260926 20:39:25.496675 25365 tablet_bootstrap.cc:492] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08: No bootstrap required, opened a new log
I20260926 20:39:25.496706 25363 ts_tablet_manager.cc:1397] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260926 20:39:25.496825 25364 ts_tablet_manager.cc:1397] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260926 20:39:25.496948 25365 ts_tablet_manager.cc:1397] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260926 20:39:25.505213 25363 raft_consensus.cc:348] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } }
I20260926 20:39:25.505674 25363 raft_consensus.cc:374] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:25.505772 25363 raft_consensus.cc:729] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be72ed9e76f3461b80a3fe51cdabedc7, State: Initialized, Role: FOLLOWER
I20260926 20:39:25.506461 25363 consensus_queue.cc:260] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } }
I20260926 20:39:25.509325 25204 heartbeater.cc:503] Master 127.23.44.254:33483 was elected leader, sending a full tablet report...
I20260926 20:39:25.509240 25365 raft_consensus.cc:348] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } }
I20260926 20:39:25.509177 25364 raft_consensus.cc:348] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } }
I20260926 20:39:25.509630 25364 raft_consensus.cc:374] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:25.509644 25365 raft_consensus.cc:374] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:25.509729 25364 raft_consensus.cc:729] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c547b182647b99c7818bd53fec93e, State: Initialized, Role: FOLLOWER
I20260926 20:39:25.509729 25365 raft_consensus.cc:729] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44d6886f29a2411bbbd4f4b94d139a08, State: Initialized, Role: FOLLOWER
I20260926 20:39:25.510028 25363 ts_tablet_manager.cc:1428] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260926 20:39:25.510430 25365 consensus_queue.cc:260] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } }
I20260926 20:39:25.510699 25364 consensus_queue.cc:260] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } }
I20260926 20:39:25.514067 25364 ts_tablet_manager.cc:1428] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e: Time spent starting tablet: real 0.017s user 0.011s sys 0.004s
I20260926 20:39:25.518898 25344 heartbeater.cc:503] Master 127.23.44.254:33483 was elected leader, sending a full tablet report...
I20260926 20:39:25.519852 25365 ts_tablet_manager.cc:1428] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08: Time spent starting tablet: real 0.023s user 0.003s sys 0.017s
W20260926 20:39:25.547710 25205 tablet.cc:2369] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:39:25.613235 25345 tablet.cc:2369] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:39:25.741619 25369 raft_consensus.cc:482] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:39:25.741961 25369 raft_consensus.cc:504] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } }
I20260926 20:39:25.743721 25369 leader_election.cc:288] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44d6886f29a2411bbbd4f4b94d139a08 (127.23.44.195:45423), 984c547b182647b99c7818bd53fec93e (127.23.44.193:36953)
W20260926 20:39:25.748754 25066 tablet.cc:2369] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:39:25.749850 25371 raft_consensus.cc:482] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:39:25.750311 25371 raft_consensus.cc:504] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } }
I20260926 20:39:25.752455 25371 leader_election.cc:288] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44d6886f29a2411bbbd4f4b94d139a08 (127.23.44.195:45423), be72ed9e76f3461b80a3fe51cdabedc7 (127.23.44.194:39963)
I20260926 20:39:25.754005 25359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.744177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:46977 (local address 127.23.44.195:45423)
0926 20:39:25.744395 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:25.744405 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:25.745426 (+ 1021us) server_negotiation.cc:407] Connection header received
0926 20:39:25.745569 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:25.745573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:25.745652 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:25.745746 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:25.746596 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.747309 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.751491 (+ 4182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.752771 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.752911 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.752978 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:25.753580 (+ 602us) server_negotiation.cc:299] Negotiation successful
0926 20:39:25.753736 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:39:25.754520 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.744312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:47385 (local address 127.23.44.193:36953)
0926 20:39:25.744594 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:25.744603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:25.745286 (+ 683us) server_negotiation.cc:407] Connection header received
0926 20:39:25.745479 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:25.745488 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:25.745574 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:25.745653 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:25.746431 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.747337 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.751339 (+ 4002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.752666 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.752859 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.752944 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:25.753902 (+ 958us) server_negotiation.cc:299] Negotiation successful
0926 20:39:25.754036 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:39:25.754935 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.744169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:45423 (local address 127.23.44.194:46977)
0926 20:39:25.745287 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:25.745306 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:25.745447 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:25.745784 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:25.745790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:25.745905 (+ 115us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:25.746417 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.746429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.747471 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.747476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.751308 (+ 3832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.751323 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.753160 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.753167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.753379 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.753415 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 20:39:25.754278 (+ 863us) client_negotiation.cc:240] Negotiation successful
0926 20:39:25.754428 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"mutex_wait_us":577,"spinlock_wait_cycles":26368,"thread_start_us":656,"threads_started":1}
I20260926 20:39:25.756228 25372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.744156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:36953 (local address 127.23.44.194:47385)
0926 20:39:25.745159 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:25.745175 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:25.745309 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:25.745709 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:25.745717 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:25.745743 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:25.746267 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.746279 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.747506 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.747509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.751143 (+ 3634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.751159 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.753565 (+ 2406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.753571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.753726 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.753756 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 20:39:25.754897 (+ 1141us) client_negotiation.cc:240] Negotiation successful
0926 20:39:25.755752 (+ 855us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":72,"thread_start_us":614,"threads_started":1}
I20260926 20:39:25.756450 25299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a739def14b457895d5885e9504e3df" candidate_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44d6886f29a2411bbbd4f4b94d139a08" is_pre_election: true
I20260926 20:39:25.757153 25299 raft_consensus.cc:2454] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be72ed9e76f3461b80a3fe51cdabedc7 in term 0.
I20260926 20:39:25.757256 25019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a739def14b457895d5885e9504e3df" candidate_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c547b182647b99c7818bd53fec93e" is_pre_election: true
I20260926 20:39:25.758028 25087 leader_election.cc:302] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44d6886f29a2411bbbd4f4b94d139a08, be72ed9e76f3461b80a3fe51cdabedc7; no voters:
I20260926 20:39:25.757907 25019 raft_consensus.cc:2454] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be72ed9e76f3461b80a3fe51cdabedc7 in term 0.
I20260926 20:39:25.758966 25369 raft_consensus.cc:2781] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:39:25.759320 25369 raft_consensus.cc:482] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:39:25.759456 25369 raft_consensus.cc:3037] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:25.764165 25369 raft_consensus.cc:504] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } }
I20260926 20:39:25.765412 25019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a739def14b457895d5885e9504e3df" candidate_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c547b182647b99c7818bd53fec93e"
I20260926 20:39:25.765681 25019 raft_consensus.cc:3037] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:25.766284 25359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.753577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:36019 (local address 127.23.44.195:45423)
0926 20:39:25.754239 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:25.754245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:25.758055 (+ 3810us) server_negotiation.cc:407] Connection header received
0926 20:39:25.758456 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:25.758462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:25.758968 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:25.759200 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:25.760512 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.761672 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.763742 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.764992 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.765114 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.765183 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:25.765863 (+ 680us) server_negotiation.cc:299] Negotiation successful
0926 20:39:25.765991 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":3531,"threads_started":1}
I20260926 20:39:25.766288 25374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.752846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:45423 (local address 127.23.44.193:36019)
0926 20:39:25.757961 (+ 5115us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:25.757979 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:25.758172 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:25.759620 (+ 1448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:25.759630 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:25.759663 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:25.760294 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.760310 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.761794 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.761801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.763555 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.763570 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.765410 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.765416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.765640 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.765678 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 20:39:25.765789 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 20:39:25.765930 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4968,"thread_start_us":6527,"threads_started":1}
I20260926 20:39:25.767161 25298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a739def14b457895d5885e9504e3df" candidate_uuid: "984c547b182647b99c7818bd53fec93e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44d6886f29a2411bbbd4f4b94d139a08" is_pre_election: true
I20260926 20:39:25.767469 25298 raft_consensus.cc:2454] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c547b182647b99c7818bd53fec93e in term 0.
I20260926 20:39:25.768056 25362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.753905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:37581 (local address 127.23.44.194:39963)
0926 20:39:25.754143 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:25.754151 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:25.760430 (+ 6279us) server_negotiation.cc:407] Connection header received
0926 20:39:25.760642 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:25.760648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:25.760744 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:25.760847 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:25.761520 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.763079 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.765559 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.766910 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.767085 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.767168 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:25.767650 (+ 482us) server_negotiation.cc:299] Negotiation successful
0926 20:39:25.767793 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:39:25.768816 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.753066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:39963 (local address 127.23.44.193:37581)
0926 20:39:25.760306 (+ 7240us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:25.760332 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:25.760477 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:25.760876 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:25.760881 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:25.760907 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:25.761262 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.761273 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.763229 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.763234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.765401 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:25.765412 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.767148 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.767157 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:25.767372 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.767406 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 20:39:25.767567 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 20:39:25.767714 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7091,"thread_start_us":6788,"threads_started":1}
I20260926 20:39:25.768790 24947 leader_election.cc:302] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44d6886f29a2411bbbd4f4b94d139a08, 984c547b182647b99c7818bd53fec93e; no voters:
I20260926 20:39:25.769133 25019 raft_consensus.cc:2454] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be72ed9e76f3461b80a3fe51cdabedc7 in term 1.
I20260926 20:39:25.769327 25371 raft_consensus.cc:2741] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260926 20:39:25.769549 25299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a739def14b457895d5885e9504e3df" candidate_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44d6886f29a2411bbbd4f4b94d139a08"
I20260926 20:39:25.769767 25299 raft_consensus.cc:3037] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:25.770268 25154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90a739def14b457895d5885e9504e3df" candidate_uuid: "984c547b182647b99c7818bd53fec93e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" is_pre_election: true
I20260926 20:39:25.770829 25154 raft_consensus.cc:2379] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c547b182647b99c7818bd53fec93e in current term 1: Already voted for candidate be72ed9e76f3461b80a3fe51cdabedc7 in this term.
I20260926 20:39:25.772765 25085 leader_election.cc:302] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c547b182647b99c7818bd53fec93e, be72ed9e76f3461b80a3fe51cdabedc7; no voters:
I20260926 20:39:25.772943 25299 raft_consensus.cc:2454] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be72ed9e76f3461b80a3fe51cdabedc7 in term 1.
I20260926 20:39:25.773411 25369 leader_election.cc:288] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [CANDIDATE]: Term 1 election: Requested vote from peers 44d6886f29a2411bbbd4f4b94d139a08 (127.23.44.195:45423), 984c547b182647b99c7818bd53fec93e (127.23.44.193:36953)
I20260926 20:39:25.773522 25377 raft_consensus.cc:2781] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:25.773746 25377 raft_consensus.cc:686] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 1 LEADER]: Becoming Leader. State: Replica: be72ed9e76f3461b80a3fe51cdabedc7, State: Running, Role: LEADER
I20260926 20:39:25.774324 25377 consensus_queue.cc:237] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } }
I20260926 20:39:25.780112 24856 catalog_manager.cc:5495] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to be72ed9e76f3461b80a3fe51cdabedc7 (127.23.44.194). New cstate: current_term: 1 leader_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } health_report { overall_health: HEALTHY } } }
I20260926 20:39:25.841468 25362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:25.834375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51876 (local address 127.23.44.194:39963)
0926 20:39:25.834598 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:25.834606 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:25.834858 (+ 252us) server_negotiation.cc:407] Connection header received
0926 20:39:25.835142 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:25.835146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:25.835223 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:25.835356 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:39:25.835752 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.836486 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.838300 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:25.838816 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:25.838931 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:25.839626 (+ 695us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:39:25.840471 (+ 845us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:39:25.840764 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:25.841045 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 20:39:25.841122 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 20:39:25.850251 25019 raft_consensus.cc:1261] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [term 1 FOLLOWER]: Refusing update from remote peer be72ed9e76f3461b80a3fe51cdabedc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:25.850255 25299 raft_consensus.cc:1261] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08 [term 1 FOLLOWER]: Refusing update from remote peer be72ed9e76f3461b80a3fe51cdabedc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:25.851179 25369 consensus_queue.cc:1035] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:25.851629 25377 consensus_queue.cc:1035] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:25.888980 24857 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:25.889300 24857 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:25.894033 24857 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782: Sending DeleteTablet for 3 replicas of tablet 90a739def14b457895d5885e9504e3df
I20260926 20:39:25.895434 25279 tablet_service.cc:1514] Processing DeleteTablet for tablet 90a739def14b457895d5885e9504e3df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:25 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:43152
I20260926 20:39:25.895452 24995 tablet_service.cc:1514] Processing DeleteTablet for tablet 90a739def14b457895d5885e9504e3df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:25 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:45558
I20260926 20:39:25.895486 25134 tablet_service.cc:1514] Processing DeleteTablet for tablet 90a739def14b457895d5885e9504e3df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:25 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:51870
I20260926 20:39:25.898243 24857 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:39:25.899009 24857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:39:25.908370 25391 tablet_replica.cc:330] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08: stopping tablet replica
I20260926 20:39:25.909866 25279 tablet_service.cc:1467] Processing CreateTablet for tablet 57d88d90532044c0ada843921fc3be73 (DEFAULT_TABLE table=test-table [id=6755c515603e4e7e9b9cdc94a520a2cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:25.909874 24994 tablet_service.cc:1467] Processing CreateTablet for tablet 57d88d90532044c0ada843921fc3be73 (DEFAULT_TABLE table=test-table [id=6755c515603e4e7e9b9cdc94a520a2cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:25.910377 25279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57d88d90532044c0ada843921fc3be73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:25.910917 24994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57d88d90532044c0ada843921fc3be73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:25.913619 25133 tablet_service.cc:1467] Processing CreateTablet for tablet 57d88d90532044c0ada843921fc3be73 (DEFAULT_TABLE table=test-table [id=6755c515603e4e7e9b9cdc94a520a2cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:25.914212 25133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57d88d90532044c0ada843921fc3be73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:25.917603 25365 tablet_bootstrap.cc:492] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08: Bootstrap starting.
I20260926 20:39:25.919407 25365 tablet_bootstrap.cc:654] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:25.922741 25364 tablet_bootstrap.cc:492] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e: Bootstrap starting.
I20260926 20:39:25.923485 25363 tablet_bootstrap.cc:492] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7: Bootstrap starting.
I20260926 20:39:25.924214 25365 tablet_bootstrap.cc:492] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08: No bootstrap required, opened a new log
I20260926 20:39:25.924345 25365 ts_tablet_manager.cc:1397] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260926 20:39:25.924642 25364 tablet_bootstrap.cc:654] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:25.925201 25363 tablet_bootstrap.cc:654] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:25.925256 25365 raft_consensus.cc:348] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:25.925474 25365 raft_consensus.cc:374] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:25.925590 25365 raft_consensus.cc:729] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44d6886f29a2411bbbd4f4b94d139a08, State: Initialized, Role: FOLLOWER
I20260926 20:39:25.927470 25365 consensus_queue.cc:260] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:25.928194 25365 ts_tablet_manager.cc:1428] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260926 20:39:25.936499 25393 tablet_replica.cc:330] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7: stopping tablet replica
I20260926 20:39:25.936673 25391 raft_consensus.cc:2229] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:25.936945 25393 raft_consensus.cc:2229] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:39:25.937654 25393 raft_consensus.cc:2258] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:25.937738 25391 raft_consensus.cc:2258] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:25.938473 25392 tablet_replica.cc:330] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e: stopping tablet replica
I20260926 20:39:25.938896 25392 raft_consensus.cc:2229] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:25.939345 25364 tablet_bootstrap.cc:492] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e: No bootstrap required, opened a new log
I20260926 20:39:25.939504 25364 ts_tablet_manager.cc:1397] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.006s
I20260926 20:39:25.939662 25392 raft_consensus.cc:2258] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:25.940095 25391 ts_tablet_manager.cc:1905] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:25.941588 25392 ts_tablet_manager.cc:1905] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:25.942047 25393 ts_tablet_manager.cc:1905] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:25.943058 25363 tablet_bootstrap.cc:492] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7: No bootstrap required, opened a new log
I20260926 20:39:25.946063 25363 ts_tablet_manager.cc:1397] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7: Time spent bootstrapping tablet: real 0.023s user 0.012s sys 0.004s
I20260926 20:39:25.947214 25391 ts_tablet_manager.cc:1918] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:39:25.947410 25391 log.cc:1196] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/90a739def14b457895d5885e9504e3df
I20260926 20:39:25.947921 25391 ts_tablet_manager.cc:1939] T 90a739def14b457895d5885e9504e3df P 44d6886f29a2411bbbd4f4b94d139a08: Deleting consensus metadata
I20260926 20:39:25.948057 25363 raft_consensus.cc:348] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:25.948732 25363 raft_consensus.cc:374] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:25.948994 25363 raft_consensus.cc:729] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be72ed9e76f3461b80a3fe51cdabedc7, State: Initialized, Role: FOLLOWER
I20260926 20:39:25.948976 25364 raft_consensus.cc:348] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:25.949244 25364 raft_consensus.cc:374] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:25.949357 25364 raft_consensus.cc:729] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c547b182647b99c7818bd53fec93e, State: Initialized, Role: FOLLOWER
I20260926 20:39:25.949812 25392 ts_tablet_manager.cc:1918] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:39:25.949748 25364 consensus_queue.cc:260] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:25.949977 25392 log.cc:1196] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/90a739def14b457895d5885e9504e3df
I20260926 20:39:25.950507 25392 ts_tablet_manager.cc:1939] T 90a739def14b457895d5885e9504e3df P 984c547b182647b99c7818bd53fec93e: Deleting consensus metadata
I20260926 20:39:25.951462 24844 catalog_manager.cc:4841] TS 44d6886f29a2411bbbd4f4b94d139a08 (127.23.44.195:45423): tablet 90a739def14b457895d5885e9504e3df (table test-table [id=40d69176f7f64aa0b5285d36233af762]) successfully deleted
I20260926 20:39:25.950405 25363 consensus_queue.cc:260] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:25.952718 25364 ts_tablet_manager.cc:1428] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e: Time spent starting tablet: real 0.013s user 0.002s sys 0.001s
I20260926 20:39:25.952759 24842 catalog_manager.cc:4841] TS 984c547b182647b99c7818bd53fec93e (127.23.44.193:36953): tablet 90a739def14b457895d5885e9504e3df (table test-table [id=40d69176f7f64aa0b5285d36233af762]) successfully deleted
I20260926 20:39:25.953117 25363 ts_tablet_manager.cc:1428] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260926 20:39:25.958894 25393 ts_tablet_manager.cc:1918] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:39:25.959156 25393 log.cc:1196] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/90a739def14b457895d5885e9504e3df
I20260926 20:39:25.960242 25393 ts_tablet_manager.cc:1939] T 90a739def14b457895d5885e9504e3df P be72ed9e76f3461b80a3fe51cdabedc7: Deleting consensus metadata
I20260926 20:39:25.961915 24842 catalog_manager.cc:4841] TS be72ed9e76f3461b80a3fe51cdabedc7 (127.23.44.194:39963): tablet 90a739def14b457895d5885e9504e3df (table test-table [id=40d69176f7f64aa0b5285d36233af762]) successfully deleted
I20260926 20:39:26.157395 25389 raft_consensus.cc:482] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:39:26.157706 25389 raft_consensus.cc:504] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:26.158733 25389 leader_election.cc:288] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 984c547b182647b99c7818bd53fec93e (127.23.44.193:36953), 44d6886f29a2411bbbd4f4b94d139a08 (127.23.44.195:45423)
I20260926 20:39:26.159291 25019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57d88d90532044c0ada843921fc3be73" candidate_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c547b182647b99c7818bd53fec93e" is_pre_election: true
I20260926 20:39:26.159348 25299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57d88d90532044c0ada843921fc3be73" candidate_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44d6886f29a2411bbbd4f4b94d139a08" is_pre_election: true
I20260926 20:39:26.159541 25019 raft_consensus.cc:2454] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be72ed9e76f3461b80a3fe51cdabedc7 in term 0.
I20260926 20:39:26.159612 25299 raft_consensus.cc:2454] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be72ed9e76f3461b80a3fe51cdabedc7 in term 0.
I20260926 20:39:26.160097 25085 leader_election.cc:302] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 44d6886f29a2411bbbd4f4b94d139a08, 984c547b182647b99c7818bd53fec93e, be72ed9e76f3461b80a3fe51cdabedc7; no voters:
I20260926 20:39:26.160953 25389 raft_consensus.cc:2781] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:39:26.161098 25389 raft_consensus.cc:482] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:39:26.161202 25389 raft_consensus.cc:3037] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:26.165021 25389 raft_consensus.cc:504] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:26.165851 25389 leader_election.cc:288] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [CANDIDATE]: Term 1 election: Requested vote from peers 984c547b182647b99c7818bd53fec93e (127.23.44.193:36953), 44d6886f29a2411bbbd4f4b94d139a08 (127.23.44.195:45423)
I20260926 20:39:26.166493 25299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57d88d90532044c0ada843921fc3be73" candidate_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44d6886f29a2411bbbd4f4b94d139a08"
I20260926 20:39:26.166502 25019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57d88d90532044c0ada843921fc3be73" candidate_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c547b182647b99c7818bd53fec93e"
I20260926 20:39:26.166741 25019 raft_consensus.cc:3037] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:26.166761 25299 raft_consensus.cc:3037] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:26.169258 25299 raft_consensus.cc:2454] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be72ed9e76f3461b80a3fe51cdabedc7 in term 1.
I20260926 20:39:26.169847 25087 leader_election.cc:302] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44d6886f29a2411bbbd4f4b94d139a08, be72ed9e76f3461b80a3fe51cdabedc7; no voters:
I20260926 20:39:26.170243 25389 raft_consensus.cc:2781] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:26.170327 25019 raft_consensus.cc:2454] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be72ed9e76f3461b80a3fe51cdabedc7 in term 1.
I20260926 20:39:26.170644 25389 raft_consensus.cc:686] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [term 1 LEADER]: Becoming Leader. State: Replica: be72ed9e76f3461b80a3fe51cdabedc7, State: Running, Role: LEADER
I20260926 20:39:26.171104 25389 consensus_queue.cc:237] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:26.175325 24857 catalog_manager.cc:5495] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to be72ed9e76f3461b80a3fe51cdabedc7 (127.23.44.194). New cstate: current_term: 1 leader_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } health_report { overall_health: UNKNOWN } } }
Sep 26 20:39:26 dist-test-slave-4d5t krb5kdc[24813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455163, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:26.257757 25403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:26.240025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55892 (local address 127.23.44.254:33483)
0926 20:39:26.240917 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:26.240927 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:26.241338 (+ 411us) server_negotiation.cc:407] Connection header received
0926 20:39:26.241542 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:26.241552 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:26.241662 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:26.241870 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:26.243403 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.244442 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.246427 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.246891 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.247058 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:26.250939 (+ 3881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:26.250948 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:26.250957 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:26.251000 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:26.253977 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:26.255092 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:26.255098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:26.255102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:26.255316 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:26.256052 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:26.256058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:26.256061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:26.256359 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:26.256522 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:26.257016 (+ 494us) server_negotiation.cc:299] Negotiation successful
0926 20:39:26.257308 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":668,"thread_start_us":406,"threads_started":1}
I20260926 20:39:26.257882 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:26.239931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33483 (local address 127.0.0.1:55892)
0926 20:39:26.241189 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:26.241209 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:26.241378 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:26.241933 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:26.241938 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:26.242770 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:26.243020 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:26.243030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.244622 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.244630 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:26.246258 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:26.246271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.247087 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.247093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:26.247292 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:26.248068 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:26.248088 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:26.250698 (+ 2610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:26.254242 (+ 3544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:26.254259 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:26.254268 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:26.254956 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:26.255749 (+ 793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:26.255754 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:26.255758 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:26.255930 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:26.256506 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:26.256515 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:26.256741 (+ 226us) client_negotiation.cc:769] Sending connection context
0926 20:39:26.256922 (+ 181us) client_negotiation.cc:240] Negotiation successful
0926 20:39:26.257438 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"mutex_wait_us":194,"thread_start_us":869,"threads_started":1}
I20260926 20:39:26.264381 24857 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:55892:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 20:39:26.265655 24857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:39:26.273286 24995 tablet_service.cc:1467] Processing CreateTablet for tablet fecb788b70094743afd8893d4317add0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=07933a3e867c49c293bb73d6a2570a46]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:39:26.273609 25279 tablet_service.cc:1467] Processing CreateTablet for tablet fecb788b70094743afd8893d4317add0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=07933a3e867c49c293bb73d6a2570a46]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:39:26.273851 24995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fecb788b70094743afd8893d4317add0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:26.273896 25134 tablet_service.cc:1467] Processing CreateTablet for tablet fecb788b70094743afd8893d4317add0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=07933a3e867c49c293bb73d6a2570a46]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:39:26.274092 25279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fecb788b70094743afd8893d4317add0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:26.274312 25134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fecb788b70094743afd8893d4317add0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:26.281489 25365 tablet_bootstrap.cc:492] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08: Bootstrap starting.
I20260926 20:39:26.283900 25365 tablet_bootstrap.cc:654] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:26.285072 25364 tablet_bootstrap.cc:492] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e: Bootstrap starting.
I20260926 20:39:26.286908 25364 tablet_bootstrap.cc:654] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:26.288518 25363 tablet_bootstrap.cc:492] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7: Bootstrap starting.
I20260926 20:39:26.289968 25364 tablet_bootstrap.cc:492] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e: No bootstrap required, opened a new log
I20260926 20:39:26.290155 25364 ts_tablet_manager.cc:1397] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 20:39:26.290367 25363 tablet_bootstrap.cc:654] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:26.290994 25364 raft_consensus.cc:348] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:26.291229 25364 raft_consensus.cc:374] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:26.291313 25364 raft_consensus.cc:729] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c547b182647b99c7818bd53fec93e, State: Initialized, Role: FOLLOWER
I20260926 20:39:26.291595 25364 consensus_queue.cc:260] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:26.292299 25371 tablet_replica.cc:439] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } }
I20260926 20:39:26.293088 25365 tablet_bootstrap.cc:492] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08: No bootstrap required, opened a new log
I20260926 20:39:26.293283 25365 ts_tablet_manager.cc:1397] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.004s
I20260926 20:39:26.293556 25371 tablet_replica.cc:442] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:26.293998 25364 ts_tablet_manager.cc:1428] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260926 20:39:26.294094 25365 raft_consensus.cc:348] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:26.294351 25365 raft_consensus.cc:374] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:26.294430 25365 raft_consensus.cc:729] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44d6886f29a2411bbbd4f4b94d139a08, State: Initialized, Role: FOLLOWER
I20260926 20:39:26.294692 25365 consensus_queue.cc:260] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:26.295300 25365 ts_tablet_manager.cc:1428] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260926 20:39:26.295192 25370 tablet_replica.cc:439] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } }
I20260926 20:39:26.295539 25370 tablet_replica.cc:442] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:26.296407 25363 tablet_bootstrap.cc:492] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7: No bootstrap required, opened a new log
I20260926 20:39:26.296968 25363 ts_tablet_manager.cc:1397] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.003s
I20260926 20:39:26.297987 25363 raft_consensus.cc:348] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:26.298293 25363 raft_consensus.cc:374] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:26.298411 25363 raft_consensus.cc:729] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be72ed9e76f3461b80a3fe51cdabedc7, State: Initialized, Role: FOLLOWER
I20260926 20:39:26.298804 25363 consensus_queue.cc:260] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:26.299415 25389 tablet_replica.cc:439] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } }
I20260926 20:39:26.299763 25389 tablet_replica.cc:442] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:26.300335 25363 ts_tablet_manager.cc:1428] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260926 20:39:26.485292 25371 raft_consensus.cc:482] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:39:26.485543 25371 raft_consensus.cc:504] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:26.486318 25371 leader_election.cc:288] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be72ed9e76f3461b80a3fe51cdabedc7 (127.23.44.194:39963), 44d6886f29a2411bbbd4f4b94d139a08 (127.23.44.195:45423)
I20260926 20:39:26.486860 25154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fecb788b70094743afd8893d4317add0" candidate_uuid: "984c547b182647b99c7818bd53fec93e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" is_pre_election: true
I20260926 20:39:26.486925 25299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fecb788b70094743afd8893d4317add0" candidate_uuid: "984c547b182647b99c7818bd53fec93e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44d6886f29a2411bbbd4f4b94d139a08" is_pre_election: true
I20260926 20:39:26.487114 25154 raft_consensus.cc:2454] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c547b182647b99c7818bd53fec93e in term 0.
I20260926 20:39:26.487232 25299 raft_consensus.cc:2454] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c547b182647b99c7818bd53fec93e in term 0.
I20260926 20:39:26.487564 24945 leader_election.cc:302] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c547b182647b99c7818bd53fec93e, be72ed9e76f3461b80a3fe51cdabedc7; no voters:
I20260926 20:39:26.487879 25371 raft_consensus.cc:2781] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:39:26.488011 25371 raft_consensus.cc:482] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:39:26.488145 25371 raft_consensus.cc:3037] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:26.490782 25371 raft_consensus.cc:504] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:26.491509 25371 leader_election.cc:288] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [CANDIDATE]: Term 1 election: Requested vote from peers be72ed9e76f3461b80a3fe51cdabedc7 (127.23.44.194:39963), 44d6886f29a2411bbbd4f4b94d139a08 (127.23.44.195:45423)
I20260926 20:39:26.492137 25154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fecb788b70094743afd8893d4317add0" candidate_uuid: "984c547b182647b99c7818bd53fec93e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be72ed9e76f3461b80a3fe51cdabedc7"
I20260926 20:39:26.492201 25299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fecb788b70094743afd8893d4317add0" candidate_uuid: "984c547b182647b99c7818bd53fec93e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44d6886f29a2411bbbd4f4b94d139a08"
I20260926 20:39:26.492401 25154 raft_consensus.cc:3037] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:26.492421 25299 raft_consensus.cc:3037] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:26.495389 25154 raft_consensus.cc:2454] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c547b182647b99c7818bd53fec93e in term 1.
I20260926 20:39:26.495375 25299 raft_consensus.cc:2454] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c547b182647b99c7818bd53fec93e in term 1.
I20260926 20:39:26.495879 24945 leader_election.cc:302] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 44d6886f29a2411bbbd4f4b94d139a08, 984c547b182647b99c7818bd53fec93e, be72ed9e76f3461b80a3fe51cdabedc7; no voters:
I20260926 20:39:26.496263 25371 raft_consensus.cc:2781] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:26.496835 25371 raft_consensus.cc:686] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [term 1 LEADER]: Becoming Leader. State: Replica: 984c547b182647b99c7818bd53fec93e, State: Running, Role: LEADER
I20260926 20:39:26.497288 25371 consensus_queue.cc:237] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } }
I20260926 20:39:26.499343 25408 tablet_replica.cc:439] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e: TxnStatusTablet state changed. Reason: New leader 984c547b182647b99c7818bd53fec93e. Latest consensus state: current_term: 1 leader_uuid: "984c547b182647b99c7818bd53fec93e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } }
I20260926 20:39:26.499742 25408 tablet_replica.cc:442] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:39:26.500344 25410 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 20:39:26.500609 25410 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 20:39:26.501258 24857 catalog_manager.cc:5495] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c547b182647b99c7818bd53fec93e (127.23.44.193). New cstate: current_term: 1 leader_uuid: "984c547b182647b99c7818bd53fec93e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } health_report { overall_health: UNKNOWN } } }
I20260926 20:39:26.546162 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:26.536611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:36953 (local address 127.0.0.1:45574)
0926 20:39:26.536859 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:26.536881 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:26.537043 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:26.537943 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:26.537948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:26.537963 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:39:26.538179 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:26.538194 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.540366 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.540373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:26.542367 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:26.542387 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.543874 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.543881 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:26.544122 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:26.544213 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:39:26.545556 (+ 1343us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:39:26.545582 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 20:39:26.545642 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 20:39:26.545822 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 20:39:26.547336 25413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:26.536626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45574 (local address 127.23.44.193:36953)
0926 20:39:26.537399 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:26.537406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:26.537431 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:39:26.537539 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:26.537545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:26.537692 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:26.537903 (+ 211us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:39:26.538663 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.540147 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.542567 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.543677 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.544436 (+ 759us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:26.544597 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:39:26.545422 (+ 825us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:39:26.546746 (+ 1324us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:26.546800 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 20:39:26.546892 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":373,"threads_started":1}
I20260926 20:39:26.552886 25154 raft_consensus.cc:1261] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7 [term 1 FOLLOWER]: Refusing update from remote peer 984c547b182647b99c7818bd53fec93e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:26.553048 25299 raft_consensus.cc:1261] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08 [term 1 FOLLOWER]: Refusing update from remote peer 984c547b182647b99c7818bd53fec93e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:26.553714 25371 consensus_queue.cc:1035] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [LEADER]: Connected to new peer: Peer: permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:26.554389 25371 consensus_queue.cc:1035] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e [LEADER]: Connected to new peer: Peer: permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:26.578044 25389 tablet_replica.cc:439] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7: TxnStatusTablet state changed. Reason: New leader 984c547b182647b99c7818bd53fec93e. Latest consensus state: current_term: 1 leader_uuid: "984c547b182647b99c7818bd53fec93e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } }
I20260926 20:39:26.578464 25389 tablet_replica.cc:442] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:26.578675 25370 tablet_replica.cc:439] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08: TxnStatusTablet state changed. Reason: New leader 984c547b182647b99c7818bd53fec93e. Latest consensus state: current_term: 1 leader_uuid: "984c547b182647b99c7818bd53fec93e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } }
I20260926 20:39:26.578933 25370 tablet_replica.cc:442] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:26.581231 25414 mvcc.cc:205] Tried to move back new op lower bound from 7333704362191499264 to 7333704361976782848. Current Snapshot: MvccSnapshot[applied={T|T < 7333704362191499264}]
I20260926 20:39:26.585628 25408 tablet_replica.cc:439] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "984c547b182647b99c7818bd53fec93e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } }
I20260926 20:39:26.585947 25408 tablet_replica.cc:442] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:39:26.585856 25415 tablet_replica.cc:439] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "984c547b182647b99c7818bd53fec93e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } }
I20260926 20:39:26.586396 25415 tablet_replica.cc:442] T fecb788b70094743afd8893d4317add0 P 984c547b182647b99c7818bd53fec93e: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:39:26.590375 25385 consensus_queue.cc:1035] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:26.599818 25370 tablet_replica.cc:439] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "984c547b182647b99c7818bd53fec93e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } }
I20260926 20:39:26.600164 25370 tablet_replica.cc:442] T fecb788b70094743afd8893d4317add0 P 44d6886f29a2411bbbd4f4b94d139a08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:26.606606 25389 tablet_replica.cc:439] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "984c547b182647b99c7818bd53fec93e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984c547b182647b99c7818bd53fec93e" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36953 } } peers { permanent_uuid: "be72ed9e76f3461b80a3fe51cdabedc7" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39963 } } peers { permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 } } }
I20260926 20:39:26.606953 25389 tablet_replica.cc:442] T fecb788b70094743afd8893d4317add0 P be72ed9e76f3461b80a3fe51cdabedc7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:26.615779 25389 consensus_queue.cc:1035] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44d6886f29a2411bbbd4f4b94d139a08" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:26.641813 25413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:26.631144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:55619 (local address 127.23.44.193:36953)
0926 20:39:26.631886 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:26.631896 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:26.636535 (+ 4639us) server_negotiation.cc:407] Connection header received
0926 20:39:26.636785 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:26.636790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:26.636869 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:26.636975 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:39:26.637834 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.638665 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.639997 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.640426 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.640591 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:26.641049 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:39:26.641148 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:39:26.641254 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:26.641397 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 20:39:26.641477 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571}
I20260926 20:39:26.641987 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:26.630182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:36953 (local address 127.23.44.194:55619)
0926 20:39:26.636413 (+ 6231us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:26.636431 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:26.636635 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:26.636988 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:26.636992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:26.637003 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:39:26.637677 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:26.637687 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.638825 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.638832 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:26.639857 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:26.639868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.640644 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.640653 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:26.640793 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:26.640853 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:39:26.641254 (+ 401us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:39:26.641279 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 20:39:26.641376 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 20:39:26.641509 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6025,"thread_start_us":5872,"threads_started":1}
I20260926 20:39:26.698346 25433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:26.678818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:39963 (local address 127.23.44.193:49583)
0926 20:39:26.691224 (+ 12406us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:26.691247 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:26.691405 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:26.692616 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:26.692622 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:26.692635 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:39:26.693010 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:26.693023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.694287 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.694294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:26.696246 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:26.696262 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.697095 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.697101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:26.697269 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:26.697370 (+ 101us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:39:26.697851 (+ 481us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:39:26.697861 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 20:39:26.697929 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 20:39:26.698028 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12188,"thread_start_us":11946,"threads_started":1}
I20260926 20:39:26.698415 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:26.678714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:49583 (local address 127.23.44.194:39963)
0926 20:39:26.692055 (+ 13341us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:26.692061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:26.692086 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:39:26.692157 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:26.692162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:26.692255 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:26.692356 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:39:26.693187 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.694148 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.696401 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.696923 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.697071 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:26.697604 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:39:26.697746 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:39:26.697839 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:26.697957 (+ 118us) server_negotiation.cc:299] Negotiation successful
0926 20:39:26.698036 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13193,"thread_start_us":12942,"threads_started":1}
I20260926 20:39:26.733294 24857 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:26.733598 24857 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:26.737071 24857 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91816fa7f5674802837097dd41ab1782: Sending DeleteTablet for 3 replicas of tablet 57d88d90532044c0ada843921fc3be73
I20260926 20:39:26.737892 24995 tablet_service.cc:1514] Processing DeleteTablet for tablet 57d88d90532044c0ada843921fc3be73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:26 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:45558
I20260926 20:39:26.737953 25134 tablet_service.cc:1514] Processing DeleteTablet for tablet 57d88d90532044c0ada843921fc3be73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:26 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:51870
I20260926 20:39:26.738096 25279 tablet_service.cc:1514] Processing DeleteTablet for tablet 57d88d90532044c0ada843921fc3be73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:26 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:43152
I20260926 20:39:26.748123 25437 tablet_replica.cc:330] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08: stopping tablet replica
I20260926 20:39:26.749177 25437 raft_consensus.cc:2229] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:26.750123 25437 raft_consensus.cc:2258] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:26.751513 25436 tablet_replica.cc:330] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7: stopping tablet replica
I20260926 20:39:26.752753 25437 ts_tablet_manager.cc:1905] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:26.753093 25436 raft_consensus.cc:2229] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:39:26.753688 25436 raft_consensus.cc:2258] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:26.754033 25435 tablet_replica.cc:330] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e: stopping tablet replica
I20260926 20:39:26.754221 25435 raft_consensus.cc:2229] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:26.754594 25435 raft_consensus.cc:2258] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:26.756896 25435 ts_tablet_manager.cc:1905] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:26.757331 25436 ts_tablet_manager.cc:1905] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:26.762454 25437 ts_tablet_manager.cc:1918] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:39:26.762615 25437 log.cc:1196] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/57d88d90532044c0ada843921fc3be73
I20260926 20:39:26.763114 25437 ts_tablet_manager.cc:1939] T 57d88d90532044c0ada843921fc3be73 P 44d6886f29a2411bbbd4f4b94d139a08: Deleting consensus metadata
I20260926 20:39:26.764395 24844 catalog_manager.cc:4841] TS 44d6886f29a2411bbbd4f4b94d139a08 (127.23.44.195:45423): tablet 57d88d90532044c0ada843921fc3be73 (table test-table [id=6755c515603e4e7e9b9cdc94a520a2cb]) successfully deleted
I20260926 20:39:26.765942 25436 ts_tablet_manager.cc:1918] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:39:26.766098 25436 log.cc:1196] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/57d88d90532044c0ada843921fc3be73
I20260926 20:39:26.766650 25436 ts_tablet_manager.cc:1939] T 57d88d90532044c0ada843921fc3be73 P be72ed9e76f3461b80a3fe51cdabedc7: Deleting consensus metadata
I20260926 20:39:26.767949 24842 catalog_manager.cc:4841] TS be72ed9e76f3461b80a3fe51cdabedc7 (127.23.44.194:39963): tablet 57d88d90532044c0ada843921fc3be73 (table test-table [id=6755c515603e4e7e9b9cdc94a520a2cb]) successfully deleted
I20260926 20:39:26.768544 25435 ts_tablet_manager.cc:1918] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:39:26.768743 25435 log.cc:1196] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/57d88d90532044c0ada843921fc3be73
I20260926 20:39:26.769222 25435 ts_tablet_manager.cc:1939] T 57d88d90532044c0ada843921fc3be73 P 984c547b182647b99c7818bd53fec93e: Deleting consensus metadata
I20260926 20:39:26.770279 24842 catalog_manager.cc:4841] TS 984c547b182647b99c7818bd53fec93e (127.23.44.193:36953): tablet 57d88d90532044c0ada843921fc3be73 (table test-table [id=6755c515603e4e7e9b9cdc94a520a2cb]) successfully deleted
Sep 26 20:39:26 dist-test-slave-4d5t krb5kdc[24813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455165, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:39:26.784129 25413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:26.752623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45588 (local address 127.23.44.193:36953)
0926 20:39:26.752843 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:26.752850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:26.762095 (+ 9245us) server_negotiation.cc:407] Connection header received
0926 20:39:26.762588 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:26.762596 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:26.762737 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:26.762834 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:26.764958 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.766087 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.770513 (+ 4426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.771154 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.771277 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:26.776467 (+ 5190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:26.776490 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:26.776512 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:26.776588 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:26.780698 (+ 4110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:26.781437 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:26.781446 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:26.781472 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:26.781646 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:26.782018 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:26.782024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:26.782027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:26.782467 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:26.782795 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:26.783462 (+ 667us) server_negotiation.cc:299] Negotiation successful
0926 20:39:26.783705 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260926 20:39:26.785168 25060 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45588
I20260926 20:39:26.807505 25441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:26.788030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55904 (local address 127.23.44.254:33483)
0926 20:39:26.788526 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:26.788533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:26.789383 (+ 850us) server_negotiation.cc:407] Connection header received
0926 20:39:26.789721 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:26.789728 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:26.789810 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:26.789924 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:26.791907 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.793414 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.795149 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.795753 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.795906 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:26.798886 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:26.798900 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:26.798905 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:26.798951 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:26.803515 (+ 4564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:26.804416 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:26.804423 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:26.804427 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:26.804703 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:26.805153 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:26.805161 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:26.805165 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:26.805790 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:26.806208 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:26.806768 (+ 560us) server_negotiation.cc:299] Negotiation successful
0926 20:39:26.807069 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":250,"threads_started":1}
W20260926 20:39:26.808125 24857 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55904
I20260926 20:39:26.810276 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 24929
I20260926 20:39:26.832450 25413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:26.811681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45598 (local address 127.23.44.193:36953)
0926 20:39:26.811891 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:26.811898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:26.819941 (+ 8043us) server_negotiation.cc:407] Connection header received
0926 20:39:26.820154 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:26.820160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:26.820244 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:26.820368 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:26.821701 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.822456 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.823692 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.824177 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.824331 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:26.826882 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:26.826888 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:26.826893 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:26.826939 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:26.829752 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:26.830417 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:26.830423 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:26.830429 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:26.830550 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:26.830931 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:26.830936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:26.830940 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:26.831227 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:26.831498 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:26.831812 (+ 314us) server_negotiation.cc:299] Negotiation successful
0926 20:39:26.832044 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260926 20:39:26.833057 25060 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45598
W20260926 20:39:26.833662 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 24929: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:39:26.833851 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 24929
I20260926 20:39:26.851763 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 25069
Sep 26 20:39:26 dist-test-slave-4d5t krb5kdc[24813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455165, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:39:26.869609 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:26.852700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.23.44.194:39963)
0926 20:39:26.852941 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:26.852947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:26.853019 (+ 72us) server_negotiation.cc:407] Connection header received
0926 20:39:26.853238 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:26.853250 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:26.853313 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:26.853417 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:26.854557 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.855269 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.856498 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.857000 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.857139 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:26.862124 (+ 4985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:26.862145 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:26.862161 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:26.862207 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:26.866288 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:26.867201 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:26.867210 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:26.867212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:26.867408 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:26.867839 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:26.867843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:26.867846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:26.868530 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:26.868862 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:26.869162 (+ 300us) server_negotiation.cc:299] Negotiation successful
0926 20:39:26.869349 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
W20260926 20:39:26.870440 25199 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51892
W20260926 20:39:26.872779 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 25069: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:39:26.872947 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 25069
I20260926 20:39:26.896723 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 25208
Sep 26 20:39:26 dist-test-slave-4d5t krb5kdc[24813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455165, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:39:26.914495 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:26.897757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43162 (local address 127.23.44.195:45423)
0926 20:39:26.898349 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:26.898355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:26.898378 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:39:26.898444 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:26.898449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:26.898524 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:26.898657 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:26.900304 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.901200 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.902597 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:26.903040 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:26.903147 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:26.907805 (+ 4658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:26.907822 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:26.907839 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:26.907883 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:26.911138 (+ 3255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:26.911858 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:26.911865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:26.911868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:26.912025 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:26.912352 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:26.912356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:26.912358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:26.912875 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:26.913137 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:26.913841 (+ 704us) server_negotiation.cc:299] Negotiation successful
0926 20:39:26.914191 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":332,"threads_started":1}
W20260926 20:39:26.915222 25339 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43162
W20260926 20:39:26.915980 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 25208: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:39:26.916186 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 25208
I20260926 20:39:26.933758 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 24826
I20260926 20:39:26.934990 24898 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:27.050911 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 24826
2026-09-26T20:39:27Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5311 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:39:27 dist-test-slave-4d5t krb5kdc[25448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:39:27 dist-test-slave-4d5t krb5kdc[25448](info): set up 2 sockets
Sep 26 20:39:27 dist-test-slave-4d5t krb5kdc[25448](info): commencing operation
krb5kdc: starting...
W20260926 20:39:28.173018 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:39:28 dist-test-slave-4d5t krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:39:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:39:28.372692 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:36845
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:36845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:28.694173 25461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:28.694543 25461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:28.694717 25461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:28.702217 25461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:28.702384 25461 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 20:39:28.702478 25461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:28.702600 25461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:28.702683 25461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:28.702773 25461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:39:28.712867 25461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:36845
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:36845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:28.714290 25461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:28.715610 25461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:28.728350 25466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:28.728384 25467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:28.728336 25469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:28.729408 25461 server_base.cc:1034] running on GCE node
I20260926 20:39:28.730214 25461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:28.731688 25461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:28.733000 25461 hybrid_clock.cc:648] HybridClock initialized: now 1790455168732911 us; error 74 us; skew 500 ppm
I20260926 20:39:28.733322 25461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:28 dist-test-slave-4d5t krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455168, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:28.736829 25461 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:39:28.738547 25461 webserver.cc:460] Webserver started at http://127.23.44.254:39455/ using document root <none> and password file <none>
I20260926 20:39:28.739382 25461 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:28.739480 25461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:28.739833 25461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:28.743302 25461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "7c3c6dadd6a9482489e7bfc5b212fa81"
format_stamp: "Formatted at 2026-09-26 20:39:28 on dist-test-slave-4d5t"
server_key: "db6f5e9b67f4b75b5fe158c15fed53f9"
server_key_iv: "6f226436e84e21b734980b4dcf1abe63"
server_key_version: "encryptionkey@0"
I20260926 20:39:28.744230 25461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "7c3c6dadd6a9482489e7bfc5b212fa81"
format_stamp: "Formatted at 2026-09-26 20:39:28 on dist-test-slave-4d5t"
server_key: "db6f5e9b67f4b75b5fe158c15fed53f9"
server_key_iv: "6f226436e84e21b734980b4dcf1abe63"
server_key_version: "encryptionkey@0"
I20260926 20:39:28.750499 25461 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260926 20:39:28.754488 25476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:28.756438 25461 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260926 20:39:28.756860 25461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "7c3c6dadd6a9482489e7bfc5b212fa81"
format_stamp: "Formatted at 2026-09-26 20:39:28 on dist-test-slave-4d5t"
server_key: "db6f5e9b67f4b75b5fe158c15fed53f9"
server_key_iv: "6f226436e84e21b734980b4dcf1abe63"
server_key_version: "encryptionkey@0"
I20260926 20:39:28.757148 25461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:28.777958 25461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:28.782630 25461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:28.782932 25461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:28.806779 25461 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:36845
I20260926 20:39:28.806792 25537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:36845 every 8 connection(s)
I20260926 20:39:28.808203 25461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:28.812139 25538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:28.812671 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 25461
I20260926 20:39:28.812894 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:28.813598 23731 external_mini_cluster.cc:1428] Setting key f14574b14dde9d7175cb72eb75c779d3
I20260926 20:39:28.826864 25538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81: Bootstrap starting.
Sep 26 20:39:28 dist-test-slave-4d5t krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:28.830236 25538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:28.831666 25538 log.cc:824] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:28.835150 25538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81: No bootstrap required, opened a new log
I20260926 20:39:28.839787 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:28.815872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49676 (local address 127.23.44.254:36845)
0926 20:39:28.816841 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:28.816857 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:28.816911 (+ 54us) server_negotiation.cc:407] Connection header received
0926 20:39:28.818871 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:28.818904 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:28.819458 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:28.819952 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:28.821388 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:28.823431 (+ 2043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:28.824892 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:28.825983 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:28.826177 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:28.830148 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:28.830160 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:28.830180 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:28.830246 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:28.834285 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:28.835581 (+ 1296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:28.835592 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:28.835604 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:28.835875 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:28.836364 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:28.836369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:28.836372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:28.837149 (+ 777us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:28.837479 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:28.837881 (+ 402us) server_negotiation.cc:299] Negotiation successful
0926 20:39:28.838355 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":457,"threads_started":1}
I20260926 20:39:28.845599 25538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3c6dadd6a9482489e7bfc5b212fa81" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36845 } }
I20260926 20:39:28.845970 25538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:28.846035 25538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3c6dadd6a9482489e7bfc5b212fa81, State: Initialized, Role: FOLLOWER
I20260926 20:39:28.846619 25538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3c6dadd6a9482489e7bfc5b212fa81" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36845 } }
I20260926 20:39:28.846843 25538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:28.847035 25538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:28.847234 25538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:28.849609 25538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3c6dadd6a9482489e7bfc5b212fa81" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36845 } }
I20260926 20:39:28.850078 25538 leader_election.cc:302] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c3c6dadd6a9482489e7bfc5b212fa81; no voters:
I20260926 20:39:28.850672 25538 leader_election.cc:288] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:28.850858 25543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:28.851565 25543 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [term 1 LEADER]: Becoming Leader. State: Replica: 7c3c6dadd6a9482489e7bfc5b212fa81, State: Running, Role: LEADER
I20260926 20:39:28.852090 25543 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3c6dadd6a9482489e7bfc5b212fa81" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36845 } }
I20260926 20:39:28.852829 25538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:39:28.855463 25545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c3c6dadd6a9482489e7bfc5b212fa81. Latest consensus state: current_term: 1 leader_uuid: "7c3c6dadd6a9482489e7bfc5b212fa81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3c6dadd6a9482489e7bfc5b212fa81" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36845 } } }
I20260926 20:39:28.855901 25545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:28.857028 25544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c3c6dadd6a9482489e7bfc5b212fa81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3c6dadd6a9482489e7bfc5b212fa81" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36845 } } }
I20260926 20:39:28.857378 25544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:28.858551 25550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:28.866061 25550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:28.876358 25550 catalog_manager.cc:1347] Generated new cluster ID: 6b9b78dae4b7428a91de2354fb409a5b
I20260926 20:39:28.876654 25550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:28.895382 25550 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:28.896991 25550 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:28.904693 25550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81: Generated new TSK 0
I20260926 20:39:28.905650 25550 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:39:28.960865 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:36845
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:29.319780 25564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:29.320115 25564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:29.320317 25564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:29.328065 25564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:29.328202 25564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:29.328378 25564 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 20:39:29.328465 25564 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:39:29.328624 25564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:39:29.339743 25564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.44.254:36845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:29.341266 25564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:29.342694 25564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:29.357724 25569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:29.358021 25570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:29.358484 25572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:29.358939 25564 server_base.cc:1034] running on GCE node
I20260926 20:39:29.359558 25564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:29.360538 25564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:29.361819 25564 hybrid_clock.cc:648] HybridClock initialized: now 1790455169361767 us; error 48 us; skew 500 ppm
I20260926 20:39:29.362221 25564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:29 dist-test-slave-4d5t krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455169, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:29.370718 25564 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:39:29.373060 25564 webserver.cc:460] Webserver started at http://127.23.44.193:33917/ using document root <none> and password file <none>
I20260926 20:39:29.373900 25564 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:29.374058 25564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:29.374406 25564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:29.377483 25564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "0ed1853bf1c249a993ebc0e5da8161eb"
format_stamp: "Formatted at 2026-09-26 20:39:29 on dist-test-slave-4d5t"
server_key: "600a40991f3d44f94e45905d2be93941"
server_key_iv: "ec98cbde954dc93b5a1b3c53330367d7"
server_key_version: "encryptionkey@0"
I20260926 20:39:29.378345 25564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "0ed1853bf1c249a993ebc0e5da8161eb"
format_stamp: "Formatted at 2026-09-26 20:39:29 on dist-test-slave-4d5t"
server_key: "600a40991f3d44f94e45905d2be93941"
server_key_iv: "ec98cbde954dc93b5a1b3c53330367d7"
server_key_version: "encryptionkey@0"
I20260926 20:39:29.383740 25564 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:39:29.387426 25579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:29.389523 25564 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260926 20:39:29.389816 25564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "0ed1853bf1c249a993ebc0e5da8161eb"
format_stamp: "Formatted at 2026-09-26 20:39:29 on dist-test-slave-4d5t"
server_key: "600a40991f3d44f94e45905d2be93941"
server_key_iv: "ec98cbde954dc93b5a1b3c53330367d7"
server_key_version: "encryptionkey@0"
I20260926 20:39:29.390239 25564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:29.415061 25564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:29.419457 25564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:29.419755 25564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:29.422542 25564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:29.422668 25564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:29.422791 25564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:29.422886 25564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:39:29 dist-test-slave-4d5t krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455169, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:29.448756 25588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:29.423136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:46303 (local address 127.23.44.254:36845)
0926 20:39:29.423838 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:29.423842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:29.425482 (+ 1640us) server_negotiation.cc:407] Connection header received
0926 20:39:29.428290 (+ 2808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:29.428301 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:29.428430 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:29.428535 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:29.431698 (+ 3163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.432542 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.435257 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.435761 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.435915 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:29.441588 (+ 5673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:29.441594 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:29.441597 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:29.441635 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:29.444537 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:29.445802 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:29.445809 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:29.445812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:29.445975 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:29.446611 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:29.446619 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:29.446623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:29.447043 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:29.447222 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:29.448047 (+ 825us) server_negotiation.cc:299] Negotiation successful
0926 20:39:29.448308 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":503,"threads_started":1}
I20260926 20:39:29.449894 25589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:29.423919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:36845 (local address 127.23.44.193:46303)
0926 20:39:29.425216 (+ 1297us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:29.425292 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:29.427785 (+ 2493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:29.429233 (+ 1448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:29.429252 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:29.430191 (+ 939us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:29.431427 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:29.431459 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.432746 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.432751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:29.435092 (+ 2341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:29.435105 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.436067 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.436078 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:29.436473 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:29.437606 (+ 1133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:29.437660 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:29.441306 (+ 3646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:29.444769 (+ 3463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:29.444780 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:29.444806 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:29.445617 (+ 811us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:29.446145 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:29.446152 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:29.446157 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:29.446458 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:29.447211 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:29.447220 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:29.447630 (+ 410us) client_negotiation.cc:769] Sending connection context
0926 20:39:29.447928 (+ 298us) client_negotiation.cc:240] Negotiation successful
0926 20:39:29.448590 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":478,"threads_started":1}
I20260926 20:39:29.480104 25564 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:44155
I20260926 20:39:29.480146 25699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:44155 every 8 connection(s)
I20260926 20:39:29.481659 25564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:39:29.482197 25588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:29.460883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:36363 (local address 127.23.44.254:36845)
0926 20:39:29.461092 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:29.461100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:29.462165 (+ 1065us) server_negotiation.cc:407] Connection header received
0926 20:39:29.463015 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:29.463021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:29.463132 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:29.463243 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:29.465539 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.466911 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.470397 (+ 3486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.470851 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.470967 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:29.474343 (+ 3376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:29.474353 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:29.474363 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:29.474423 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:29.478693 (+ 4270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:29.479946 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:29.479955 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:29.479961 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:29.480146 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:29.480697 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:29.480702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:29.480704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:29.480988 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:29.481152 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:29.481576 (+ 424us) server_negotiation.cc:299] Negotiation successful
0926 20:39:29.481814 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 20:39:29.482535 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:29.460918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:36845 (local address 127.23.44.193:36363)
0926 20:39:29.462044 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:29.462069 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:29.462729 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:29.463323 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:29.463336 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:29.464475 (+ 1139us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:29.465295 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:29.465319 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.467137 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.467161 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:29.470207 (+ 3046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:29.470231 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.471083 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.471091 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:29.471293 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:29.472275 (+ 982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:29.472310 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:29.474011 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:29.478953 (+ 4942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:29.478959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:29.478965 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:29.479732 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:29.480420 (+ 688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:29.480424 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:29.480426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:29.480548 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:29.481114 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:29.481118 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:29.481322 (+ 204us) client_negotiation.cc:769] Sending connection context
0926 20:39:29.481461 (+ 139us) client_negotiation.cc:240] Negotiation successful
0926 20:39:29.481903 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":485,"threads_started":1}
I20260926 20:39:29.483614 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 25564
I20260926 20:39:29.483811 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:39:29.484644 23731 external_mini_cluster.cc:1428] Setting key 4a206ab335176ed3646fba7701c3136b
I20260926 20:39:29.486284 25700 heartbeater.cc:347] Connected to a master server at 127.23.44.254:36845
I20260926 20:39:29.486545 25700 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:29.487468 25700 heartbeater.cc:511] Master 127.23.44.254:36845 requested a full tablet report, sending...
I20260926 20:39:29.489526 25492 ts_manager.cc:194] Registered new tserver with Master: 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193:44155)
I20260926 20:39:29.491099 25492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:46303
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:39:29.531634 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:36845
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:29.867167 25704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:29.867553 25704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:29.867738 25704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:29.876056 25704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:29.876240 25704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:29.876384 25704 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 20:39:29.876469 25704 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:39:29.876617 25704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:39:29.886956 25704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.44.254:36845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:29.888329 25704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:29.889706 25704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:29.902878 25709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:29.902849 25710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:29.902925 25712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:29.904155 25704 server_base.cc:1034] running on GCE node
I20260926 20:39:29.904878 25704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:29.905910 25704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:29.907222 25704 hybrid_clock.cc:648] HybridClock initialized: now 1790455169907140 us; error 42 us; skew 500 ppm
I20260926 20:39:29.907756 25704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:29 dist-test-slave-4d5t krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455169, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:29.911559 25704 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:39:29.913636 25704 webserver.cc:460] Webserver started at http://127.23.44.194:34923/ using document root <none> and password file <none>
I20260926 20:39:29.914525 25704 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:29.914709 25704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:29.915060 25704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:29.918311 25704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "313df5efe7ed4ba7ae0d4f1981d3322d"
format_stamp: "Formatted at 2026-09-26 20:39:29 on dist-test-slave-4d5t"
server_key: "0486e2d24bb39541e770b086ea540ed3"
server_key_iv: "a2caf90719fd755be04719686a2649fa"
server_key_version: "encryptionkey@0"
I20260926 20:39:29.919142 25704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "313df5efe7ed4ba7ae0d4f1981d3322d"
format_stamp: "Formatted at 2026-09-26 20:39:29 on dist-test-slave-4d5t"
server_key: "0486e2d24bb39541e770b086ea540ed3"
server_key_iv: "a2caf90719fd755be04719686a2649fa"
server_key_version: "encryptionkey@0"
I20260926 20:39:29.924754 25704 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260926 20:39:29.928505 25719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:29.930081 25704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:39:29.930269 25704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "313df5efe7ed4ba7ae0d4f1981d3322d"
format_stamp: "Formatted at 2026-09-26 20:39:29 on dist-test-slave-4d5t"
server_key: "0486e2d24bb39541e770b086ea540ed3"
server_key_iv: "a2caf90719fd755be04719686a2649fa"
server_key_version: "encryptionkey@0"
I20260926 20:39:29.930493 25704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:29.942289 25704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:29.947541 25704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:29.947957 25704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:29.952015 25704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:29.952184 25704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:29.952322 25704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:29.952445 25704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:39:29 dist-test-slave-4d5t krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455169, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:29.975666 25588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:29.952649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:44511 (local address 127.23.44.254:36845)
0926 20:39:29.952859 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:29.952867 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:29.955086 (+ 2219us) server_negotiation.cc:407] Connection header received
0926 20:39:29.957179 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:29.957185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:29.957276 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:29.957385 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:29.959786 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.960955 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.963701 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.964262 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.964404 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:29.968966 (+ 4562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:29.968971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:29.968974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:29.969009 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:29.971580 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:29.972353 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:29.972361 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:29.972364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:29.972485 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:29.972984 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:29.972992 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:29.972995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:29.973416 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:29.973630 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:29.974688 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0926 20:39:29.975054 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 20:39:29.976325 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:29.953822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:36845 (local address 127.23.44.194:44511)
0926 20:39:29.954864 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:29.954917 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:29.956889 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:29.957624 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:29.957633 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:29.958507 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:29.959579 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:29.959610 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.961102 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.961109 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:29.963464 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:29.963476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.964756 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.964761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:29.964952 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:29.965835 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:29.965861 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:29.968724 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:29.971750 (+ 3026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:29.971761 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:29.971781 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:29.972232 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:29.972626 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:29.972632 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:29.972635 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:29.972847 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:29.973563 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:29.973572 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:29.974167 (+ 595us) client_negotiation.cc:769] Sending connection context
0926 20:39:29.974534 (+ 367us) client_negotiation.cc:240] Negotiation successful
0926 20:39:29.975015 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":467,"threads_started":1}
I20260926 20:39:30.006007 25588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:29.988912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:41581 (local address 127.23.44.254:36845)
0926 20:39:29.989176 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:29.989186 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:29.989935 (+ 749us) server_negotiation.cc:407] Connection header received
0926 20:39:29.990242 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:29.990249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:29.990327 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:29.990439 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:29.991508 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.992328 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.994606 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.995209 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.995392 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:29.997966 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:29.997975 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:29.997983 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:29.998034 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:30.002072 (+ 4038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:30.003368 (+ 1296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:30.003376 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:30.003382 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:30.003599 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:30.004157 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:30.004163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:30.004167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:30.004618 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:30.004810 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:30.005393 (+ 583us) server_negotiation.cc:299] Negotiation successful
0926 20:39:30.005622 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 20:39:30.006142 25777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:29.989003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:36845 (local address 127.23.44.194:41581)
0926 20:39:29.989792 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:29.989823 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:29.990018 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:29.990471 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:29.990476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:29.991033 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:29.991350 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:29.991360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.992478 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.992484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:29.994430 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:29.994450 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:29.995425 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:29.995432 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:29.995655 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:29.996384 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:29.996403 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:29.997603 (+ 1200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:30.002249 (+ 4646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:30.002257 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:30.002264 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:30.003125 (+ 861us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:30.003761 (+ 636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:30.003770 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:30.003782 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:30.003995 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:30.004889 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:30.004896 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:30.005120 (+ 224us) client_negotiation.cc:769] Sending connection context
0926 20:39:30.005298 (+ 178us) client_negotiation.cc:240] Negotiation successful
0926 20:39:30.005654 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":397,"threads_started":1}
I20260926 20:39:30.022632 25704 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:41997
I20260926 20:39:30.022691 25838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:41997 every 8 connection(s)
I20260926 20:39:30.024047 25704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:39:30.026304 25839 heartbeater.cc:347] Connected to a master server at 127.23.44.254:36845
I20260926 20:39:30.026497 25839 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:30.027110 25839 heartbeater.cc:511] Master 127.23.44.254:36845 requested a full tablet report, sending...
I20260926 20:39:30.028410 25492 ts_manager.cc:194] Registered new tserver with Master: 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194:41997)
I20260926 20:39:30.029248 25492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:44511
I20260926 20:39:30.032028 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 25704
I20260926 20:39:30.032199 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:39:30.032752 23731 external_mini_cluster.cc:1428] Setting key 2eacc8f86199bf6bcd5a9aacc07e24f9
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:39:30.076892 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:36845
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:30.418447 25843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:30.418762 25843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:30.418872 25843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:30.426390 25843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:30.426506 25843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:30.426592 25843 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 20:39:30.426644 25843 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:39:30.426713 25843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:39:30.436781 25843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.44.254:36845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:30.438021 25843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:30.439302 25843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:30.453197 25848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:30.453229 25849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:30.453197 25851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:30.454496 25843 server_base.cc:1034] running on GCE node
I20260926 20:39:30.455138 25843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:30.456187 25843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:30.457442 25843 hybrid_clock.cc:648] HybridClock initialized: now 1790455170457391 us; error 47 us; skew 500 ppm
I20260926 20:39:30.457787 25843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:30 dist-test-slave-4d5t krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455170, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:30.461184 25843 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:39:30.462888 25843 webserver.cc:460] Webserver started at http://127.23.44.195:44771/ using document root <none> and password file <none>
I20260926 20:39:30.463644 25843 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:30.463788 25843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:30.464102 25843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:30.468101 25843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "62a55f9c956649a08162025051186e3d"
format_stamp: "Formatted at 2026-09-26 20:39:30 on dist-test-slave-4d5t"
server_key: "619588030ff29f4ff63606721168301f"
server_key_iv: "658326d960a20758993a5a59cd24c845"
server_key_version: "encryptionkey@0"
I20260926 20:39:30.469143 25843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "62a55f9c956649a08162025051186e3d"
format_stamp: "Formatted at 2026-09-26 20:39:30 on dist-test-slave-4d5t"
server_key: "619588030ff29f4ff63606721168301f"
server_key_iv: "658326d960a20758993a5a59cd24c845"
server_key_version: "encryptionkey@0"
I20260926 20:39:30.474954 25843 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:39:30.479058 25858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:30.480811 25843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 20:39:30.481060 25843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "62a55f9c956649a08162025051186e3d"
format_stamp: "Formatted at 2026-09-26 20:39:30 on dist-test-slave-4d5t"
server_key: "619588030ff29f4ff63606721168301f"
server_key_iv: "658326d960a20758993a5a59cd24c845"
server_key_version: "encryptionkey@0"
I20260926 20:39:30.481295 25843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:30.494514 25700 heartbeater.cc:503] Master 127.23.44.254:36845 was elected leader, sending a full tablet report...
I20260926 20:39:30.503394 25843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:30.508111 25843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:30.508368 25843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:30.511639 25843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:30.511782 25843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:30.511911 25843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:30.511986 25843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:39:30 dist-test-slave-4d5t krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455170, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:30.539353 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.512094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:60667 (local address 127.23.44.254:36845)
0926 20:39:30.512890 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:30.512897 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:30.514024 (+ 1127us) server_negotiation.cc:407] Connection header received
0926 20:39:30.516257 (+ 2233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:30.516264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:30.516360 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:30.516489 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:30.519202 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.520323 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.523042 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.523707 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.523888 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.530422 (+ 6534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:30.530429 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:30.530435 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:30.530483 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:30.534689 (+ 4206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:30.536006 (+ 1317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:30.536013 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:30.536017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:30.536178 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:30.536805 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:30.536811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:30.536814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:30.537537 (+ 723us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:30.537769 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:30.538677 (+ 908us) server_negotiation.cc:299] Negotiation successful
0926 20:39:30.538940 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":562,"threads_started":1}
I20260926 20:39:30.540426 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.512868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:36845 (local address 127.23.44.195:60667)
0926 20:39:30.513831 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:30.513882 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:30.515917 (+ 2035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:30.516872 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:30.516887 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:30.517878 (+ 991us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:30.518961 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.518996 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.520519 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.520530 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.522837 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.522851 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.524171 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.524190 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.524838 (+ 648us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.525924 (+ 1086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:30.525965 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:30.529979 (+ 4014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:30.535051 (+ 5072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:30.535069 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:30.535090 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:30.535826 (+ 736us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:30.536298 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:30.536305 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:30.536309 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:30.536668 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:30.537771 (+ 1103us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:30.537785 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:30.538249 (+ 464us) client_negotiation.cc:769] Sending connection context
0926 20:39:30.538548 (+ 299us) client_negotiation.cc:240] Negotiation successful
0926 20:39:30.539149 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":435,"threads_started":1}
I20260926 20:39:30.566434 25843 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:36835
I20260926 20:39:30.566520 25978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:36835 every 8 connection(s)
I20260926 20:39:30.567799 25843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:39:30.568104 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 25843
I20260926 20:39:30.568280 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:39:30.568521 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.549381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:58037 (local address 127.23.44.254:36845)
0926 20:39:30.549642 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:30.549649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:30.551217 (+ 1568us) server_negotiation.cc:407] Connection header received
0926 20:39:30.551443 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:30.551451 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:30.551570 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:30.551684 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:30.553551 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.554439 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.557285 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.557775 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.557877 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.561081 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:30.561090 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:30.561098 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:30.561149 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:30.565301 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:30.566411 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:30.566417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:30.566421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:30.566584 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:30.566986 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:30.566991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:30.566995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:30.567274 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:30.567446 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:30.567898 (+ 452us) server_negotiation.cc:299] Negotiation successful
0926 20:39:30.568148 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260926 20:39:30.568898 23731 external_mini_cluster.cc:1428] Setting key 4bbfa22925d8b565dc1c2c583b421a35
I20260926 20:39:30.570183 25939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.549706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:36845 (local address 127.23.44.195:58037)
0926 20:39:30.551060 (+ 1354us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:30.551083 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:30.551296 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:30.551781 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:30.551792 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:30.552769 (+ 977us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:30.553355 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.553378 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.554601 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.554610 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.557130 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.557147 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.557980 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.557988 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.558218 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.559126 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:30.559155 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:30.560838 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:30.565486 (+ 4648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:30.565495 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:30.565502 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:30.566231 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:30.566736 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:30.566741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:30.566744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:30.566868 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:30.567395 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:30.567400 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:30.567593 (+ 193us) client_negotiation.cc:769] Sending connection context
0926 20:39:30.569009 (+ 1416us) client_negotiation.cc:240] Negotiation successful
0926 20:39:30.569517 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1149,"thread_start_us":940,"threads_started":1}
I20260926 20:39:30.572217 25979 heartbeater.cc:347] Connected to a master server at 127.23.44.254:36845
I20260926 20:39:30.572469 25979 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:30.573199 25979 heartbeater.cc:511] Master 127.23.44.254:36845 requested a full tablet report, sending...
I20260926 20:39:30.575016 25491 ts_manager.cc:194] Registered new tserver with Master: 62a55f9c956649a08162025051186e3d (127.23.44.195:36835)
I20260926 20:39:30.575922 25491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:60667
I20260926 20:39:30.584754 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 20:39:30 dist-test-slave-4d5t krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 20:39:30 dist-test-slave-4d5t krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:30.635552 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.619601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49686 (local address 127.23.44.254:36845)
0926 20:39:30.619908 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:30.619916 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:30.620177 (+ 261us) server_negotiation.cc:407] Connection header received
0926 20:39:30.620318 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:30.620323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:30.620446 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:30.620679 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:30.622281 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.623346 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.624692 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.625234 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.625348 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.629567 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:30.629573 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:30.629577 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:30.629621 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:30.632342 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:30.633056 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:30.633067 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:30.633070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:30.633194 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:30.633694 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:30.633698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:30.633701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:30.634000 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:30.634398 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:30.634942 (+ 544us) server_negotiation.cc:299] Negotiation successful
0926 20:39:30.635144 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260926 20:39:30.640606 25491 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:39:30.647657 25491 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:39:30.672851 25995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.663750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51634 (local address 127.23.44.193:44155)
0926 20:39:30.664444 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:30.664451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:30.665044 (+ 593us) server_negotiation.cc:407] Connection header received
0926 20:39:30.665256 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:30.665265 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:30.665771 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:30.665962 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:30.666840 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.667844 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.670668 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.671607 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.671760 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.671991 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:30.672285 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 20:39:30.672480 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":387,"threads_started":1}
I20260926 20:39:30.672984 25994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.663447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44155 (local address 127.0.0.1:51634)
0926 20:39:30.664920 (+ 1473us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:30.664939 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:30.665086 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:30.666147 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:30.666150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:30.666168 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:30.666622 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.666639 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.667967 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.667975 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.670522 (+ 2547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.670532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.671789 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.671795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.672065 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.672104 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 20:39:30.672243 (+ 139us) client_negotiation.cc:240] Negotiation successful
0926 20:39:30.672508 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1356,"mutex_wait_us":105,"thread_start_us":1266,"threads_started":1}
I20260926 20:39:30.673687 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.662004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:41997 (local address 127.0.0.1:46578)
0926 20:39:30.663302 (+ 1298us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:30.663347 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:30.663564 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:30.665367 (+ 1803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:30.665377 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:30.665412 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:30.665862 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.665881 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.668157 (+ 2276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.668163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.670630 (+ 2467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.670643 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.672689 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.672696 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.672874 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.672901 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 20:39:30.673155 (+ 254us) client_negotiation.cc:240] Negotiation successful
0926 20:39:30.673294 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":112,"thread_start_us":742,"threads_started":1}
I20260926 20:39:30.674561 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.662272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46578 (local address 127.23.44.194:41997)
0926 20:39:30.664128 (+ 1856us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:30.664139 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:30.664170 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:39:30.664278 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:30.664287 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:30.664531 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:30.664775 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:30.666097 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.667948 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.670813 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.672172 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.672321 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.672506 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:30.674092 (+ 1586us) server_negotiation.cc:299] Negotiation successful
0926 20:39:30.674260 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1711,"thread_start_us":1589,"threads_started":1}
I20260926 20:39:30.676445 25630 tablet_service.cc:1467] Processing CreateTablet for tablet cd62cfafe80f4d0995df2a6f5a8de97e (DEFAULT_TABLE table=test-table [id=42f0f302aa9c4ec98d3e8157811c6b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:30.677632 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.665602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57494 (local address 127.23.44.195:36835)
0926 20:39:30.668499 (+ 2897us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:30.668508 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:30.668539 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:39:30.668660 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:30.668668 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:30.668918 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:30.669096 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:30.670092 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.671706 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.675590 (+ 3884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.676527 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.676738 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.676964 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:30.677161 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 20:39:30.677318 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2725,"thread_start_us":2519,"threads_started":1}
I20260926 20:39:30.677636 25997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.666191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:36835 (local address 127.0.0.1:57494)
0926 20:39:30.666806 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:30.666823 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:30.666920 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:30.669366 (+ 2446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:30.669371 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:30.669419 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:30.669874 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.669886 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.671891 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.671896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.675236 (+ 3340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.675266 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.676783 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.676789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.676977 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.677014 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 20:39:30.677124 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 20:39:30.677261 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":27,"thread_start_us":441,"threads_started":1}
I20260926 20:39:30.678196 25630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd62cfafe80f4d0995df2a6f5a8de97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:30.680045 25773 tablet_service.cc:1467] Processing CreateTablet for tablet cd62cfafe80f4d0995df2a6f5a8de97e (DEFAULT_TABLE table=test-table [id=42f0f302aa9c4ec98d3e8157811c6b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:30.680855 25909 tablet_service.cc:1467] Processing CreateTablet for tablet cd62cfafe80f4d0995df2a6f5a8de97e (DEFAULT_TABLE table=test-table [id=42f0f302aa9c4ec98d3e8157811c6b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:30.681496 25773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd62cfafe80f4d0995df2a6f5a8de97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:30.681953 25909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd62cfafe80f4d0995df2a6f5a8de97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:30.694029 25998 tablet_bootstrap.cc:492] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d: Bootstrap starting.
I20260926 20:39:30.697292 25999 tablet_bootstrap.cc:492] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb: Bootstrap starting.
I20260926 20:39:30.697912 25998 tablet_bootstrap.cc:654] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:30.699296 25998 log.cc:824] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:30.701215 25999 tablet_bootstrap.cc:654] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:30.702744 25999 log.cc:824] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:30.704450 26000 tablet_bootstrap.cc:492] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d: Bootstrap starting.
I20260926 20:39:30.706758 25999 tablet_bootstrap.cc:492] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb: No bootstrap required, opened a new log
I20260926 20:39:30.707056 25999 ts_tablet_manager.cc:1397] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260926 20:39:30.707260 26000 tablet_bootstrap.cc:654] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:30.708822 26000 log.cc:824] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:30.709195 25998 tablet_bootstrap.cc:492] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d: No bootstrap required, opened a new log
I20260926 20:39:30.709473 25998 ts_tablet_manager.cc:1397] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.004s
I20260926 20:39:30.711251 26000 tablet_bootstrap.cc:492] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d: No bootstrap required, opened a new log
I20260926 20:39:30.711500 26000 ts_tablet_manager.cc:1397] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260926 20:39:30.717806 25999 raft_consensus.cc:348] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.718467 25999 raft_consensus.cc:374] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:30.718686 25999 raft_consensus.cc:729] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ed1853bf1c249a993ebc0e5da8161eb, State: Initialized, Role: FOLLOWER
I20260926 20:39:30.719506 26000 raft_consensus.cc:348] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.719892 26000 raft_consensus.cc:374] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:30.720016 26000 raft_consensus.cc:729] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313df5efe7ed4ba7ae0d4f1981d3322d, State: Initialized, Role: FOLLOWER
I20260926 20:39:30.720050 25999 consensus_queue.cc:260] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.720779 26000 consensus_queue.cc:260] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.722258 25999 ts_tablet_manager.cc:1428] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260926 20:39:30.722944 25998 raft_consensus.cc:348] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.724715 25998 raft_consensus.cc:374] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:30.724907 25998 raft_consensus.cc:729] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a55f9c956649a08162025051186e3d, State: Initialized, Role: FOLLOWER
I20260926 20:39:30.725682 25998 consensus_queue.cc:260] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.727030 25839 heartbeater.cc:503] Master 127.23.44.254:36845 was elected leader, sending a full tablet report...
I20260926 20:39:30.727425 25979 heartbeater.cc:503] Master 127.23.44.254:36845 was elected leader, sending a full tablet report...
I20260926 20:39:30.728070 26000 ts_tablet_manager.cc:1428] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260926 20:39:30.728282 25998 ts_tablet_manager.cc:1428] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d: Time spent starting tablet: real 0.019s user 0.015s sys 0.001s
W20260926 20:39:30.736109 25701 tablet.cc:2369] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:39:30.776312 25840 tablet.cc:2369] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:39:30.825256 25980 tablet.cc:2369] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:39:30.843029 26006 raft_consensus.cc:482] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:39:30.843437 26006 raft_consensus.cc:504] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.845302 26006 leader_election.cc:288] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194:41997), 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193:44155)
I20260926 20:39:30.852873 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.846052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:41401 (local address 127.23.44.194:41997)
0926 20:39:30.846284 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:30.846291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:30.846312 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:39:30.846477 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:30.846482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:30.846545 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:30.846652 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:30.847239 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.848352 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.850342 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.851840 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.852041 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.852100 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:30.852496 (+ 396us) server_negotiation.cc:299] Negotiation successful
0926 20:39:30.852678 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:39:30.852933 25868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.845808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:41997 (local address 127.23.44.195:41401)
0926 20:39:30.846147 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:30.846170 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:30.846331 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:30.846818 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:30.846822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:30.846846 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:30.847050 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.847063 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.848475 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.848483 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.850135 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.850152 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.852042 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.852047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.852305 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.852339 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 20:39:30.852447 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 20:39:30.852637 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260926 20:39:30.854075 25794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd62cfafe80f4d0995df2a6f5a8de97e" candidate_uuid: "62a55f9c956649a08162025051186e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" is_pre_election: true
I20260926 20:39:30.854373 25995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.846526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:33573 (local address 127.23.44.193:44155)
0926 20:39:30.846748 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:30.846754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:30.847216 (+ 462us) server_negotiation.cc:407] Connection header received
0926 20:39:30.847470 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:30.847474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:30.847540 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:30.847607 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:30.848267 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.849357 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.852023 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.853367 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.853578 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.853630 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:30.854063 (+ 433us) server_negotiation.cc:299] Negotiation successful
0926 20:39:30.854157 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 20:39:30.854648 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.846432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44155 (local address 127.23.44.195:33573)
0926 20:39:30.847097 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:30.847114 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:30.847318 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:30.847672 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:30.847677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:30.847764 (+ 87us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:30.848111 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.848122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.849591 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.849597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.851852 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.851866 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.853714 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.853721 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.853903 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.853939 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 20:39:30.854045 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 20:39:30.854221 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":358,"threads_started":1}
I20260926 20:39:30.854641 25794 raft_consensus.cc:2454] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a55f9c956649a08162025051186e3d in term 0.
I20260926 20:39:30.855329 25862 leader_election.cc:302] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313df5efe7ed4ba7ae0d4f1981d3322d, 62a55f9c956649a08162025051186e3d; no voters:
I20260926 20:39:30.855770 26006 raft_consensus.cc:2781] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:39:30.855631 25655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd62cfafe80f4d0995df2a6f5a8de97e" candidate_uuid: "62a55f9c956649a08162025051186e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" is_pre_election: true
I20260926 20:39:30.855930 26006 raft_consensus.cc:482] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:39:30.856065 26006 raft_consensus.cc:3037] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:30.856137 25655 raft_consensus.cc:2454] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a55f9c956649a08162025051186e3d in term 0.
I20260926 20:39:30.858852 26006 raft_consensus.cc:504] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.859481 26006 leader_election.cc:288] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [CANDIDATE]: Term 1 election: Requested vote from peers 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194:41997), 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193:44155)
I20260926 20:39:30.860008 25794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd62cfafe80f4d0995df2a6f5a8de97e" candidate_uuid: "62a55f9c956649a08162025051186e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d"
I20260926 20:39:30.860101 25655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd62cfafe80f4d0995df2a6f5a8de97e" candidate_uuid: "62a55f9c956649a08162025051186e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed1853bf1c249a993ebc0e5da8161eb"
I20260926 20:39:30.860224 25794 raft_consensus.cc:3037] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:30.860345 25655 raft_consensus.cc:3037] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:30.862756 25655 raft_consensus.cc:2454] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a55f9c956649a08162025051186e3d in term 1.
I20260926 20:39:30.863067 25794 raft_consensus.cc:2454] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a55f9c956649a08162025051186e3d in term 1.
I20260926 20:39:30.863163 25862 leader_election.cc:302] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed1853bf1c249a993ebc0e5da8161eb, 62a55f9c956649a08162025051186e3d; no voters:
I20260926 20:39:30.863504 26006 raft_consensus.cc:2781] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:30.863969 26006 raft_consensus.cc:686] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [term 1 LEADER]: Becoming Leader. State: Replica: 62a55f9c956649a08162025051186e3d, State: Running, Role: LEADER
I20260926 20:39:30.864482 26006 consensus_queue.cc:237] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.869241 25493 catalog_manager.cc:5495] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 62a55f9c956649a08162025051186e3d (127.23.44.195). New cstate: current_term: 1 leader_uuid: "62a55f9c956649a08162025051186e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } health_report { overall_health: HEALTHY } } }
Sep 26 20:39:30 dist-test-slave-4d5t krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455168, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:30.940781 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.927046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49698 (local address 127.23.44.254:36845)
0926 20:39:30.927261 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:30.927269 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:30.928017 (+ 748us) server_negotiation.cc:407] Connection header received
0926 20:39:30.928208 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:30.928218 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:30.928286 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:30.928383 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:30.929593 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.930349 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.931618 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.932072 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.932178 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.935559 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:30.935563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:30.935566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:30.935602 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:30.938085 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:30.938779 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:30.938787 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:30.938791 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:30.938903 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:30.939258 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:30.939264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:30.939270 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:30.939533 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:30.939687 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:30.940062 (+ 375us) server_negotiation.cc:299] Negotiation successful
0926 20:39:30.940415 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":151,"server-negotiator.queue_time_us":83}
I20260926 20:39:30.940815 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:30.927228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:36845 (local address 127.0.0.1:49698)
0926 20:39:30.927917 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:30.927932 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:30.928058 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:30.928442 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:30.928446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:30.929176 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:30.929439 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.929448 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.930481 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.930485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.931473 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:30.931483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:30.932249 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:30.932254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:30.932361 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:30.933081 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:30.933102 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:30.935370 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:30.938231 (+ 2861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:30.938245 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:30.938253 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:30.938669 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:30.939008 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:30.939013 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:30.939017 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:30.939159 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:30.939660 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:30.939668 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:30.939825 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 20:39:30.939984 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 20:39:30.940309 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":26,"thread_start_us":382,"threads_started":1}
I20260926 20:39:30.946211 25493 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:49698:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 20:39:30.947340 25493 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:39:30.954566 25630 tablet_service.cc:1467] Processing CreateTablet for tablet c95466c4e54b4d71a49a89dac689d90a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a59d574ee12d4ad1935ece245ac3634f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:39:30.954965 25909 tablet_service.cc:1467] Processing CreateTablet for tablet c95466c4e54b4d71a49a89dac689d90a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a59d574ee12d4ad1935ece245ac3634f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:39:30.954872 25773 tablet_service.cc:1467] Processing CreateTablet for tablet c95466c4e54b4d71a49a89dac689d90a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a59d574ee12d4ad1935ece245ac3634f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:39:30.955677 25630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c95466c4e54b4d71a49a89dac689d90a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:30.955703 25909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c95466c4e54b4d71a49a89dac689d90a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:30.955708 25773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c95466c4e54b4d71a49a89dac689d90a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:30.961611 26000 tablet_bootstrap.cc:492] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d: Bootstrap starting.
I20260926 20:39:30.963006 25998 tablet_bootstrap.cc:492] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d: Bootstrap starting.
I20260926 20:39:30.963107 26000 tablet_bootstrap.cc:654] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:30.963817 25999 tablet_bootstrap.cc:492] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb: Bootstrap starting.
I20260926 20:39:30.965034 25998 tablet_bootstrap.cc:654] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:30.965394 25999 tablet_bootstrap.cc:654] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:30.967501 25999 tablet_bootstrap.cc:492] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb: No bootstrap required, opened a new log
I20260926 20:39:30.967661 25999 ts_tablet_manager.cc:1397] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 20:39:30.968338 25999 raft_consensus.cc:348] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.968523 25999 raft_consensus.cc:374] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:30.968657 25999 raft_consensus.cc:729] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ed1853bf1c249a993ebc0e5da8161eb, State: Initialized, Role: FOLLOWER
I20260926 20:39:30.968909 25999 consensus_queue.cc:260] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.969352 25999 ts_tablet_manager.cc:1428] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 20:39:30.969678 26004 tablet_replica.cc:439] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } } }
I20260926 20:39:30.969993 26004 tablet_replica.cc:442] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:30.971098 26000 tablet_bootstrap.cc:492] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d: No bootstrap required, opened a new log
I20260926 20:39:30.971271 26000 ts_tablet_manager.cc:1397] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260926 20:39:30.972103 25998 tablet_bootstrap.cc:492] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d: No bootstrap required, opened a new log
I20260926 20:39:30.972297 25998 ts_tablet_manager.cc:1397] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260926 20:39:30.972448 26000 raft_consensus.cc:348] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.972746 26000 raft_consensus.cc:374] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:30.972853 26000 raft_consensus.cc:729] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313df5efe7ed4ba7ae0d4f1981d3322d, State: Initialized, Role: FOLLOWER
I20260926 20:39:30.973145 26000 consensus_queue.cc:260] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.973491 25998 raft_consensus.cc:348] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.973742 25998 raft_consensus.cc:374] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:30.973831 25998 raft_consensus.cc:729] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a55f9c956649a08162025051186e3d, State: Initialized, Role: FOLLOWER
I20260926 20:39:30.973618 26005 tablet_replica.cc:439] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } } }
I20260926 20:39:30.973999 26005 tablet_replica.cc:442] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:30.974241 25998 consensus_queue.cc:260] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:30.974567 26000 ts_tablet_manager.cc:1428] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 20:39:30.974622 26006 tablet_replica.cc:439] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } } }
I20260926 20:39:30.974838 25998 ts_tablet_manager.cc:1428] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260926 20:39:30.974888 26006 tablet_replica.cc:442] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:31.093729 26004 raft_consensus.cc:482] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:39:31.094020 26004 raft_consensus.cc:504] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:31.095700 26004 leader_election.cc:288] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194:41997), 62a55f9c956649a08162025051186e3d (127.23.44.195:36835)
I20260926 20:39:31.102864 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:31.096181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:49553 (local address 127.23.44.194:41997)
0926 20:39:31.096413 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:31.096419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:31.096829 (+ 410us) server_negotiation.cc:407] Connection header received
0926 20:39:31.097109 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:31.097115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:31.097212 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:31.097280 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:31.098126 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.098898 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.101035 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.101839 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.102035 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:31.102123 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:31.102409 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 20:39:31.102538 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:39:31.102979 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:31.095926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:41997 (local address 127.23.44.193:49553)
0926 20:39:31.096708 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:31.096733 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:31.096907 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:31.097513 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:31.097519 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:31.097544 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:31.097960 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:31.097976 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.099045 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.099052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:31.100888 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:31.100900 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.102066 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.102073 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:31.102228 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:31.102253 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 20:39:31.102382 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 20:39:31.102526 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":86,"thread_start_us":314,"threads_started":1}
I20260926 20:39:31.103360 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:31.096825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:34623 (local address 127.23.44.195:36835)
0926 20:39:31.097042 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:31.097049 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:31.097494 (+ 445us) server_negotiation.cc:407] Connection header received
0926 20:39:31.097568 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:31.097571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:31.097628 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:31.097710 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:31.098353 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.099033 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.100937 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.101980 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.102298 (+ 318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:31.102395 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:31.102765 (+ 370us) server_negotiation.cc:299] Negotiation successful
0926 20:39:31.102922 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:39:31.103658 25794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c95466c4e54b4d71a49a89dac689d90a" candidate_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" is_pre_election: true
I20260926 20:39:31.103776 26020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:31.096631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:36835 (local address 127.23.44.193:34623)
0926 20:39:31.097340 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:31.097357 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:31.097479 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:31.097763 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:31.097766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:31.097789 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:31.098201 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:31.098209 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.099176 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.099182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:31.100773 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:31.100786 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.102374 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.102381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:31.102576 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:31.102607 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 20:39:31.102735 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 20:39:31.102917 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":45,"thread_start_us":355,"threads_started":1}
I20260926 20:39:31.103950 25794 raft_consensus.cc:2454] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ed1853bf1c249a993ebc0e5da8161eb in term 0.
I20260926 20:39:31.104457 25583 leader_election.cc:302] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed1853bf1c249a993ebc0e5da8161eb, 313df5efe7ed4ba7ae0d4f1981d3322d; no voters:
I20260926 20:39:31.104691 25929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c95466c4e54b4d71a49a89dac689d90a" candidate_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a55f9c956649a08162025051186e3d" is_pre_election: true
I20260926 20:39:31.104938 26004 raft_consensus.cc:2781] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:39:31.105027 25929 raft_consensus.cc:2454] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ed1853bf1c249a993ebc0e5da8161eb in term 0.
I20260926 20:39:31.105083 26004 raft_consensus.cc:482] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:39:31.105201 26004 raft_consensus.cc:3037] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:31.107440 26004 raft_consensus.cc:504] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:31.108141 26004 leader_election.cc:288] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [CANDIDATE]: Term 1 election: Requested vote from peers 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194:41997), 62a55f9c956649a08162025051186e3d (127.23.44.195:36835)
I20260926 20:39:31.108693 25929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c95466c4e54b4d71a49a89dac689d90a" candidate_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a55f9c956649a08162025051186e3d"
I20260926 20:39:31.108711 25794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c95466c4e54b4d71a49a89dac689d90a" candidate_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d"
I20260926 20:39:31.108906 25929 raft_consensus.cc:3037] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:31.108990 25794 raft_consensus.cc:3037] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:31.110932 25794 raft_consensus.cc:2454] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ed1853bf1c249a993ebc0e5da8161eb in term 1.
I20260926 20:39:31.110960 25929 raft_consensus.cc:2454] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ed1853bf1c249a993ebc0e5da8161eb in term 1.
I20260926 20:39:31.111340 25583 leader_election.cc:302] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed1853bf1c249a993ebc0e5da8161eb, 313df5efe7ed4ba7ae0d4f1981d3322d; no voters:
I20260926 20:39:31.111625 26004 raft_consensus.cc:2781] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:31.112104 26004 raft_consensus.cc:686] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 LEADER]: Becoming Leader. State: Replica: 0ed1853bf1c249a993ebc0e5da8161eb, State: Running, Role: LEADER
I20260926 20:39:31.112661 26004 consensus_queue.cc:237] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:31.116497 26021 tablet_replica.cc:439] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb: TxnStatusTablet state changed. Reason: New leader 0ed1853bf1c249a993ebc0e5da8161eb. Latest consensus state: current_term: 1 leader_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } } }
I20260926 20:39:31.117007 26021 tablet_replica.cc:442] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:39:31.117780 26023 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 20:39:31.118043 26023 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 20:39:31.118307 25492 catalog_manager.cc:5495] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193). New cstate: current_term: 1 leader_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } health_report { overall_health: UNKNOWN } } }
I20260926 20:39:31.171844 25995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:31.165407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51640 (local address 127.23.44.193:44155)
0926 20:39:31.165569 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:31.165573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:31.165591 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:39:31.165869 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:31.165887 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:31.165974 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:31.166110 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:39:31.166559 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.167404 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.169357 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.169827 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.169947 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:31.170648 (+ 701us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:39:31.171199 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:39:31.171307 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:31.171474 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 20:39:31.171566 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 20:39:31.171896 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:31.165189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44155 (local address 127.0.0.1:51640)
0926 20:39:31.165447 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:31.165468 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:31.165623 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:31.166168 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:31.166173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:31.166186 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:39:31.166402 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:31.166414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.167551 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.167557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:31.169196 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:31.169210 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.170104 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.170110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:31.170359 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:31.170433 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:39:31.171307 (+ 874us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:39:31.171327 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 20:39:31.171424 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 20:39:31.171585 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 20:39:31.179538 25929 raft_consensus.cc:1261] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Refusing update from remote peer 0ed1853bf1c249a993ebc0e5da8161eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:31.179566 25794 raft_consensus.cc:1261] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Refusing update from remote peer 0ed1853bf1c249a993ebc0e5da8161eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:31.180544 26021 consensus_queue.cc:1035] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:31.180902 26004 consensus_queue.cc:1035] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:31.189108 26006 tablet_replica.cc:439] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d: TxnStatusTablet state changed. Reason: New leader 0ed1853bf1c249a993ebc0e5da8161eb. Latest consensus state: current_term: 1 leader_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } } }
I20260926 20:39:31.189459 26006 tablet_replica.cc:442] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:31.192142 26026 mvcc.cc:205] Tried to move back new op lower bound from 7333704381141569536 to 7333704380881072128. Current Snapshot: MvccSnapshot[applied={T|T < 7333704381141569536}]
I20260926 20:39:31.195559 26021 tablet_replica.cc:439] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } } }
I20260926 20:39:31.195969 26021 tablet_replica.cc:442] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:39:31.203406 26006 tablet_replica.cc:439] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } } }
I20260926 20:39:31.203709 26006 tablet_replica.cc:442] T c95466c4e54b4d71a49a89dac689d90a P 62a55f9c956649a08162025051186e3d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:31.209391 26005 tablet_replica.cc:439] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d: TxnStatusTablet state changed. Reason: New leader 0ed1853bf1c249a993ebc0e5da8161eb. Latest consensus state: current_term: 1 leader_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } } }
I20260926 20:39:31.209714 26005 tablet_replica.cc:442] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:31.211297 26021 tablet_replica.cc:439] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } } }
I20260926 20:39:31.211589 26021 tablet_replica.cc:442] T c95466c4e54b4d71a49a89dac689d90a P 0ed1853bf1c249a993ebc0e5da8161eb: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:39:31.221129 26005 tablet_replica.cc:439] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } } }
I20260926 20:39:31.221418 26005 tablet_replica.cc:442] T c95466c4e54b4d71a49a89dac689d90a P 313df5efe7ed4ba7ae0d4f1981d3322d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:39:31.231968 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:31.220989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57500 (local address 127.23.44.195:36835)
0926 20:39:31.221895 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:31.221904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:31.225239 (+ 3335us) server_negotiation.cc:407] Connection header received
0926 20:39:31.225676 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:31.225685 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:31.225808 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:31.225976 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:39:31.226685 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.227512 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.229643 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.230075 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.230172 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:31.230667 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:39:31.231228 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:39:31.231344 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:31.231515 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 20:39:31.231597 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707}
I20260926 20:39:31.250538 25995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:31.244037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:59859 (local address 127.23.44.193:44155)
0926 20:39:31.244193 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:31.244198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:31.244454 (+ 256us) server_negotiation.cc:407] Connection header received
0926 20:39:31.244766 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:31.244770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:31.244849 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:31.244927 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:39:31.245691 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.246682 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.248705 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.249140 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.249240 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:31.249816 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:39:31.249944 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:39:31.250019 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:31.250216 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 20:39:31.250289 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 20:39:31.250713 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:31.243654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44155 (local address 127.23.44.195:59859)
0926 20:39:31.244324 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:31.244344 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:31.244607 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:31.244965 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:31.244970 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:31.244983 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:39:31.245353 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:31.245364 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.246818 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.246823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:31.248506 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:31.248521 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.249330 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.249337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:31.249516 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:31.249590 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:39:31.250038 (+ 448us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:39:31.250049 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 20:39:31.250134 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 20:39:31.250305 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":298,"threads_started":1}
I20260926 20:39:31.256209 26006 consensus_queue.cc:1035] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:31.261195 25655 raft_consensus.cc:1261] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Refusing update from remote peer 62a55f9c956649a08162025051186e3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:31.261893 26006 consensus_queue.cc:1035] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:31.301636 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:31.293302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:36805 (local address 127.23.44.195:36835)
0926 20:39:31.293521 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:31.293528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:31.294746 (+ 1218us) server_negotiation.cc:407] Connection header received
0926 20:39:31.294985 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:31.294990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:31.295066 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:31.295160 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:39:31.295841 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.297007 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.299247 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.300030 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.300187 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:31.300893 (+ 706us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:39:31.301037 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:39:31.301127 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:31.301261 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 20:39:31.301339 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 20:39:31.301824 26043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:31.293556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:36835 (local address 127.23.44.193:36805)
0926 20:39:31.294641 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:31.294659 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:31.294848 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:31.295175 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:31.295179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:31.295190 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:39:31.295664 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:31.295682 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.297145 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.297149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:31.299034 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:31.299060 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:31.300361 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:31.300367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:31.300599 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:31.300689 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:39:31.301132 (+ 443us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:39:31.301146 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 20:39:31.301235 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 20:39:31.301399 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"thread_start_us":756,"threads_started":1}
I20260926 20:39:31.342675 25491 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:31.342973 25491 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:31.347424 25491 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81: Sending DeleteTablet for 3 replicas of tablet cd62cfafe80f4d0995df2a6f5a8de97e
I20260926 20:39:31.349164 25909 tablet_service.cc:1514] Processing DeleteTablet for tablet cd62cfafe80f4d0995df2a6f5a8de97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:31 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:57494
I20260926 20:39:31.349161 25773 tablet_service.cc:1514] Processing DeleteTablet for tablet cd62cfafe80f4d0995df2a6f5a8de97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:31 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:46578
I20260926 20:39:31.351120 25630 tablet_service.cc:1514] Processing DeleteTablet for tablet cd62cfafe80f4d0995df2a6f5a8de97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:31 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:51634
I20260926 20:39:31.351217 25491 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:39:31.351934 25491 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:39:31.361845 25908 tablet_service.cc:1467] Processing CreateTablet for tablet 5394c1829ca3494a85f393a286d8b6ac (DEFAULT_TABLE table=test-table [id=ba1c8e8b5e7a432cb8b295e8e5c625a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:31.362403 25908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5394c1829ca3494a85f393a286d8b6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:31.362322 25629 tablet_service.cc:1467] Processing CreateTablet for tablet 5394c1829ca3494a85f393a286d8b6ac (DEFAULT_TABLE table=test-table [id=ba1c8e8b5e7a432cb8b295e8e5c625a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:31.362846 25629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5394c1829ca3494a85f393a286d8b6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:31.365139 25772 tablet_service.cc:1467] Processing CreateTablet for tablet 5394c1829ca3494a85f393a286d8b6ac (DEFAULT_TABLE table=test-table [id=ba1c8e8b5e7a432cb8b295e8e5c625a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:31.365692 25772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5394c1829ca3494a85f393a286d8b6ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:31.378973 26000 tablet_bootstrap.cc:492] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d: Bootstrap starting.
I20260926 20:39:31.380838 25999 tablet_bootstrap.cc:492] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb: Bootstrap starting.
I20260926 20:39:31.381100 26000 tablet_bootstrap.cc:654] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:31.381747 26049 tablet_replica.cc:330] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb: stopping tablet replica
I20260926 20:39:31.382489 26049 raft_consensus.cc:2229] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:31.383257 26047 tablet_replica.cc:330] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d: stopping tablet replica
I20260926 20:39:31.383301 25999 tablet_bootstrap.cc:654] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:31.385309 25998 tablet_bootstrap.cc:492] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d: Bootstrap starting.
I20260926 20:39:31.385710 26049 raft_consensus.cc:2258] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:31.385864 26047 raft_consensus.cc:2229] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:31.386591 26047 raft_consensus.cc:2258] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:31.387239 25998 tablet_bootstrap.cc:654] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:31.388509 26049 ts_tablet_manager.cc:1905] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:31.388516 26047 ts_tablet_manager.cc:1905] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:31.390584 25998 tablet_bootstrap.cc:492] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d: No bootstrap required, opened a new log
I20260926 20:39:31.390765 25998 ts_tablet_manager.cc:1397] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260926 20:39:31.391556 25998 raft_consensus.cc:348] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:31.391819 25998 raft_consensus.cc:374] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:31.391893 25998 raft_consensus.cc:729] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a55f9c956649a08162025051186e3d, State: Initialized, Role: FOLLOWER
I20260926 20:39:31.392172 25998 consensus_queue.cc:260] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:31.395520 25998 ts_tablet_manager.cc:1428] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260926 20:39:31.396122 26048 tablet_replica.cc:330] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d: stopping tablet replica
I20260926 20:39:31.396659 26048 raft_consensus.cc:2229] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:39:31.396996 25999 tablet_bootstrap.cc:492] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb: No bootstrap required, opened a new log
I20260926 20:39:31.397167 25999 ts_tablet_manager.cc:1397] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.007s
I20260926 20:39:31.397910 25999 raft_consensus.cc:348] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:31.398170 25999 raft_consensus.cc:374] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:31.398264 25999 raft_consensus.cc:729] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ed1853bf1c249a993ebc0e5da8161eb, State: Initialized, Role: FOLLOWER
I20260926 20:39:31.398514 25999 consensus_queue.cc:260] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:31.398931 26048 raft_consensus.cc:2258] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:31.399264 25999 ts_tablet_manager.cc:1428] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 20:39:31.402999 26047 ts_tablet_manager.cc:1918] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:39:31.403259 26047 log.cc:1196] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/cd62cfafe80f4d0995df2a6f5a8de97e
I20260926 20:39:31.403913 26047 ts_tablet_manager.cc:1939] T cd62cfafe80f4d0995df2a6f5a8de97e P 313df5efe7ed4ba7ae0d4f1981d3322d: Deleting consensus metadata
I20260926 20:39:31.404392 26049 ts_tablet_manager.cc:1918] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:39:31.404583 26049 log.cc:1196] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/cd62cfafe80f4d0995df2a6f5a8de97e
I20260926 20:39:31.404887 26048 ts_tablet_manager.cc:1905] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:31.405128 26049 ts_tablet_manager.cc:1939] T cd62cfafe80f4d0995df2a6f5a8de97e P 0ed1853bf1c249a993ebc0e5da8161eb: Deleting consensus metadata
I20260926 20:39:31.406188 25480 catalog_manager.cc:4841] TS 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194:41997): tablet cd62cfafe80f4d0995df2a6f5a8de97e (table test-table [id=42f0f302aa9c4ec98d3e8157811c6b1e]) successfully deleted
I20260926 20:39:31.406744 25480 catalog_manager.cc:4841] TS 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193:44155): tablet cd62cfafe80f4d0995df2a6f5a8de97e (table test-table [id=42f0f302aa9c4ec98d3e8157811c6b1e]) successfully deleted
I20260926 20:39:31.411593 26048 ts_tablet_manager.cc:1918] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:39:31.411813 26048 log.cc:1196] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/cd62cfafe80f4d0995df2a6f5a8de97e
I20260926 20:39:31.412380 26048 ts_tablet_manager.cc:1939] T cd62cfafe80f4d0995df2a6f5a8de97e P 62a55f9c956649a08162025051186e3d: Deleting consensus metadata
I20260926 20:39:31.413250 26000 tablet_bootstrap.cc:492] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d: No bootstrap required, opened a new log
I20260926 20:39:31.413481 26000 ts_tablet_manager.cc:1397] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.004s
I20260926 20:39:31.414402 25480 catalog_manager.cc:4841] TS 62a55f9c956649a08162025051186e3d (127.23.44.195:36835): tablet cd62cfafe80f4d0995df2a6f5a8de97e (table test-table [id=42f0f302aa9c4ec98d3e8157811c6b1e]) successfully deleted
I20260926 20:39:31.414366 26000 raft_consensus.cc:348] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:31.414762 26000 raft_consensus.cc:374] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:31.414855 26000 raft_consensus.cc:729] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313df5efe7ed4ba7ae0d4f1981d3322d, State: Initialized, Role: FOLLOWER
I20260926 20:39:31.415105 26000 consensus_queue.cc:260] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:31.415846 26000 ts_tablet_manager.cc:1428] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 20:39:31.479207 26008 raft_consensus.cc:482] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:39:31.479451 26008 raft_consensus.cc:504] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:31.480341 26008 leader_election.cc:288] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193:44155), 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194:41997)
I20260926 20:39:31.481407 25654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5394c1829ca3494a85f393a286d8b6ac" candidate_uuid: "62a55f9c956649a08162025051186e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" is_pre_election: true
I20260926 20:39:31.481727 25654 raft_consensus.cc:2454] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a55f9c956649a08162025051186e3d in term 0.
I20260926 20:39:31.482422 25793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5394c1829ca3494a85f393a286d8b6ac" candidate_uuid: "62a55f9c956649a08162025051186e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" is_pre_election: true
I20260926 20:39:31.482681 25793 raft_consensus.cc:2454] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a55f9c956649a08162025051186e3d in term 0.
I20260926 20:39:31.483556 25862 leader_election.cc:302] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed1853bf1c249a993ebc0e5da8161eb, 62a55f9c956649a08162025051186e3d; no voters:
I20260926 20:39:31.484275 26008 raft_consensus.cc:2781] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:39:31.484414 26008 raft_consensus.cc:482] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:39:31.484526 26008 raft_consensus.cc:3037] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:31.488384 26008 raft_consensus.cc:504] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:31.490438 25654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5394c1829ca3494a85f393a286d8b6ac" candidate_uuid: "62a55f9c956649a08162025051186e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed1853bf1c249a993ebc0e5da8161eb"
I20260926 20:39:31.490676 25654 raft_consensus.cc:3037] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:31.494808 25654 raft_consensus.cc:2454] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a55f9c956649a08162025051186e3d in term 1.
I20260926 20:39:31.495445 25862 leader_election.cc:302] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed1853bf1c249a993ebc0e5da8161eb, 62a55f9c956649a08162025051186e3d; no voters:
I20260926 20:39:31.495802 26045 raft_consensus.cc:2781] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:31.495978 26045 raft_consensus.cc:686] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [term 1 LEADER]: Becoming Leader. State: Replica: 62a55f9c956649a08162025051186e3d, State: Running, Role: LEADER
I20260926 20:39:31.496371 26045 consensus_queue.cc:237] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:31.500063 25793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5394c1829ca3494a85f393a286d8b6ac" candidate_uuid: "62a55f9c956649a08162025051186e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d"
I20260926 20:39:31.500330 25793 raft_consensus.cc:3037] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:31.503858 25793 raft_consensus.cc:2454] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a55f9c956649a08162025051186e3d in term 1.
I20260926 20:39:31.504807 26008 leader_election.cc:288] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [CANDIDATE]: Term 1 election: Requested vote from peers 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193:44155), 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194:41997)
I20260926 20:39:31.513269 25491 catalog_manager.cc:5495] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 62a55f9c956649a08162025051186e3d (127.23.44.195). New cstate: current_term: 1 leader_uuid: "62a55f9c956649a08162025051186e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } health_report { overall_health: HEALTHY } } }
I20260926 20:39:31.548602 25654 raft_consensus.cc:1261] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Refusing update from remote peer 62a55f9c956649a08162025051186e3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:31.549800 26008 consensus_queue.cc:1035] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:31.550901 25793 raft_consensus.cc:1261] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Refusing update from remote peer 62a55f9c956649a08162025051186e3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:31.551508 26008 consensus_queue.cc:1035] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:31.567929 26027 mvcc.cc:205] Tried to move back new op lower bound from 7333704382655586304 to 7333704382452551680. Current Snapshot: MvccSnapshot[applied={T|T < 7333704382655586304}]
W20260926 20:39:31.593583 26027 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 20:39:33.210881 25492 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:33.211176 25492 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:33.216352 25492 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81: Sending DeleteTablet for 3 replicas of tablet 5394c1829ca3494a85f393a286d8b6ac
I20260926 20:39:33.217837 25629 tablet_service.cc:1514] Processing DeleteTablet for tablet 5394c1829ca3494a85f393a286d8b6ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:33 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:51634
I20260926 20:39:33.218043 25909 tablet_service.cc:1514] Processing DeleteTablet for tablet 5394c1829ca3494a85f393a286d8b6ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:33 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:57494
I20260926 20:39:33.218220 25773 tablet_service.cc:1514] Processing DeleteTablet for tablet 5394c1829ca3494a85f393a286d8b6ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:33 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:46578
I20260926 20:39:33.230105 26077 tablet_replica.cc:330] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb: stopping tablet replica
I20260926 20:39:33.230528 26077 raft_consensus.cc:2229] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:33.230809 26077 raft_consensus.cc:2258] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:33.230808 26079 tablet_replica.cc:330] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d: stopping tablet replica
I20260926 20:39:33.231060 26079 raft_consensus.cc:2229] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:33.231338 26079 raft_consensus.cc:2258] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:33.231408 26078 tablet_replica.cc:330] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d: stopping tablet replica
I20260926 20:39:33.231716 26078 raft_consensus.cc:2229] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:39:33.232101 26078 raft_consensus.cc:2258] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:33.232372 26079 ts_tablet_manager.cc:1905] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:33.232995 26077 ts_tablet_manager.cc:1905] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:33.232995 26078 ts_tablet_manager.cc:1905] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:33.239639 26079 ts_tablet_manager.cc:1918] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 20:39:33.239784 26077 ts_tablet_manager.cc:1918] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 20:39:33.239774 26078 ts_tablet_manager.cc:1918] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 20:39:33.239933 26078 log.cc:1196] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/5394c1829ca3494a85f393a286d8b6ac
I20260926 20:39:33.239928 26079 log.cc:1196] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/5394c1829ca3494a85f393a286d8b6ac
I20260926 20:39:33.240073 26077 log.cc:1196] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/5394c1829ca3494a85f393a286d8b6ac
I20260926 20:39:33.240459 26079 ts_tablet_manager.cc:1939] T 5394c1829ca3494a85f393a286d8b6ac P 313df5efe7ed4ba7ae0d4f1981d3322d: Deleting consensus metadata
I20260926 20:39:33.240687 26078 ts_tablet_manager.cc:1939] T 5394c1829ca3494a85f393a286d8b6ac P 62a55f9c956649a08162025051186e3d: Deleting consensus metadata
I20260926 20:39:33.240993 26077 ts_tablet_manager.cc:1939] T 5394c1829ca3494a85f393a286d8b6ac P 0ed1853bf1c249a993ebc0e5da8161eb: Deleting consensus metadata
I20260926 20:39:33.241930 25480 catalog_manager.cc:4841] TS 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194:41997): tablet 5394c1829ca3494a85f393a286d8b6ac (table test-table [id=ba1c8e8b5e7a432cb8b295e8e5c625a0]) successfully deleted
I20260926 20:39:33.242262 25480 catalog_manager.cc:4841] TS 62a55f9c956649a08162025051186e3d (127.23.44.195:36835): tablet 5394c1829ca3494a85f393a286d8b6ac (table test-table [id=ba1c8e8b5e7a432cb8b295e8e5c625a0]) successfully deleted
I20260926 20:39:33.242481 25480 catalog_manager.cc:4841] TS 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193:44155): tablet 5394c1829ca3494a85f393a286d8b6ac (table test-table [id=ba1c8e8b5e7a432cb8b295e8e5c625a0]) successfully deleted
Sep 26 20:39:33 dist-test-slave-4d5t krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455173, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 20:39:33 dist-test-slave-4d5t krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455173, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:33.286576 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:33.268505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49712 (local address 127.23.44.254:36845)
0926 20:39:33.269196 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:33.269201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:33.269227 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:39:33.269365 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:33.269370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:33.269468 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:33.269578 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:33.271015 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.272019 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.273618 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.274148 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.274310 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:33.279545 (+ 5235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:33.279552 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:33.279557 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:33.279607 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:33.283188 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:33.284039 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:33.284043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:33.284046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:33.284178 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:33.284701 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:33.284705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:33.284708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:33.285002 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:33.285332 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:33.285814 (+ 482us) server_negotiation.cc:299] Negotiation successful
0926 20:39:33.286214 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":345,"threads_started":1}
I20260926 20:39:33.290475 25492 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:39:33.291069 25492 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:39:33.298969 25629 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe6bccdba94474289bcdaf78dc580a4 (DEFAULT_TABLE table=test-table [id=41849beed52549a98430140aaa03fa06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:33.299116 25773 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe6bccdba94474289bcdaf78dc580a4 (DEFAULT_TABLE table=test-table [id=41849beed52549a98430140aaa03fa06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:33.299472 25773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe6bccdba94474289bcdaf78dc580a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:33.299492 25629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe6bccdba94474289bcdaf78dc580a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:33.299494 25909 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe6bccdba94474289bcdaf78dc580a4 (DEFAULT_TABLE table=test-table [id=41849beed52549a98430140aaa03fa06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:33.299959 25909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe6bccdba94474289bcdaf78dc580a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:33.306416 26088 tablet_bootstrap.cc:492] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d: Bootstrap starting.
I20260926 20:39:33.308665 26088 tablet_bootstrap.cc:654] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:33.310938 26088 tablet_bootstrap.cc:492] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d: No bootstrap required, opened a new log
I20260926 20:39:33.311115 26088 ts_tablet_manager.cc:1397] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 20:39:33.311530 26089 tablet_bootstrap.cc:492] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb: Bootstrap starting.
I20260926 20:39:33.312487 26088 raft_consensus.cc:348] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.310357 26090 tablet_bootstrap.cc:492] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d: Bootstrap starting.
I20260926 20:39:33.312933 26088 raft_consensus.cc:374] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:33.313100 26088 raft_consensus.cc:729] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a55f9c956649a08162025051186e3d, State: Initialized, Role: FOLLOWER
I20260926 20:39:33.313454 26088 consensus_queue.cc:260] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.313761 26089 tablet_bootstrap.cc:654] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:33.314199 26088 ts_tablet_manager.cc:1428] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 20:39:33.314730 26090 tablet_bootstrap.cc:654] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:33.316267 26089 tablet_bootstrap.cc:492] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb: No bootstrap required, opened a new log
I20260926 20:39:33.316450 26089 ts_tablet_manager.cc:1397] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260926 20:39:33.317186 26090 tablet_bootstrap.cc:492] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d: No bootstrap required, opened a new log
I20260926 20:39:33.317404 26090 ts_tablet_manager.cc:1397] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260926 20:39:33.317405 26089 raft_consensus.cc:348] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.317683 26089 raft_consensus.cc:374] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:33.317832 26089 raft_consensus.cc:729] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ed1853bf1c249a993ebc0e5da8161eb, State: Initialized, Role: FOLLOWER
I20260926 20:39:33.318135 26089 consensus_queue.cc:260] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.318857 26090 raft_consensus.cc:348] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.319192 26090 raft_consensus.cc:374] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:33.319355 26090 raft_consensus.cc:729] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313df5efe7ed4ba7ae0d4f1981d3322d, State: Initialized, Role: FOLLOWER
I20260926 20:39:33.319406 26089 ts_tablet_manager.cc:1428] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 20:39:33.319650 26090 consensus_queue.cc:260] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.320652 26090 ts_tablet_manager.cc:1428] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 20:39:33.574186 26094 raft_consensus.cc:482] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:39:33.574563 26094 raft_consensus.cc:504] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.576424 26094 leader_election.cc:288] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193:44155), 62a55f9c956649a08162025051186e3d (127.23.44.195:36835)
I20260926 20:39:33.588006 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:33.577941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:45445 (local address 127.23.44.195:36835)
0926 20:39:33.578700 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:33.578710 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:33.579688 (+ 978us) server_negotiation.cc:407] Connection header received
0926 20:39:33.579970 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:33.579974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:33.580033 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:33.580146 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:33.581378 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.582392 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.585370 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.586775 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.586982 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:33.587056 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:33.587565 (+ 509us) server_negotiation.cc:299] Negotiation successful
0926 20:39:33.587693 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":356,"threads_started":1}
I20260926 20:39:33.587984 26095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:33.576933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44155 (local address 127.23.44.194:47733)
0926 20:39:33.577789 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:33.577827 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:33.577997 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:33.580178 (+ 2181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:33.580183 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:33.580232 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:33.581193 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:33.581230 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.582842 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.582847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:33.585372 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:33.585397 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.586921 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.586928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:33.587139 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:33.587186 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 20:39:33.587323 (+ 137us) client_negotiation.cc:240] Negotiation successful
0926 20:39:33.587487 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":1,"spinlock_wait_cycles":9088,"thread_start_us":430,"threads_started":1}
I20260926 20:39:33.588070 26097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:33.577817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:36835 (local address 127.23.44.194:45445)
0926 20:39:33.579570 (+ 1753us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:33.579595 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:33.579810 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:33.580175 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:33.580180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:33.580222 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:33.581085 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:33.581097 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.582558 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.582562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:33.585137 (+ 2575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:33.585157 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.587186 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.587192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:33.587409 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:33.587460 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 20:39:33.587547 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 20:39:33.587685 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1551,"spinlock_wait_cycles":3840,"thread_start_us":1476,"threads_started":1}
I20260926 20:39:33.588699 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:33.576936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:47733 (local address 127.23.44.193:44155)
0926 20:39:33.579852 (+ 2916us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:33.579859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:33.579883 (+ 24us) server_negotiation.cc:407] Connection header received
0926 20:39:33.579953 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:33.579958 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:33.580026 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:33.581386 (+ 1360us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:33.581577 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.582692 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.585559 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.586686 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.586920 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:33.587006 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:33.587345 (+ 339us) server_negotiation.cc:299] Negotiation successful
0926 20:39:33.587491 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2787,"thread_start_us":2650,"threads_started":1}
I20260926 20:39:33.589083 25929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe6bccdba94474289bcdaf78dc580a4" candidate_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a55f9c956649a08162025051186e3d" is_pre_election: true
I20260926 20:39:33.589385 25929 raft_consensus.cc:2454] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 313df5efe7ed4ba7ae0d4f1981d3322d in term 0.
I20260926 20:39:33.589339 25655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe6bccdba94474289bcdaf78dc580a4" candidate_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" is_pre_election: true
I20260926 20:39:33.589815 25655 raft_consensus.cc:2454] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 313df5efe7ed4ba7ae0d4f1981d3322d in term 0.
I20260926 20:39:33.590358 25723 leader_election.cc:302] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 313df5efe7ed4ba7ae0d4f1981d3322d, 62a55f9c956649a08162025051186e3d; no voters:
I20260926 20:39:33.590830 26094 raft_consensus.cc:2781] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:39:33.591099 26094 raft_consensus.cc:482] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:39:33.591264 26094 raft_consensus.cc:3037] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:33.594345 26094 raft_consensus.cc:504] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.595108 26094 leader_election.cc:288] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [CANDIDATE]: Term 1 election: Requested vote from peers 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193:44155), 62a55f9c956649a08162025051186e3d (127.23.44.195:36835)
I20260926 20:39:33.595599 25655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe6bccdba94474289bcdaf78dc580a4" candidate_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed1853bf1c249a993ebc0e5da8161eb"
I20260926 20:39:33.595722 25929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe6bccdba94474289bcdaf78dc580a4" candidate_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a55f9c956649a08162025051186e3d"
I20260926 20:39:33.595849 25655 raft_consensus.cc:3037] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:33.595986 25929 raft_consensus.cc:3037] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:33.598170 25655 raft_consensus.cc:2454] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313df5efe7ed4ba7ae0d4f1981d3322d in term 1.
I20260926 20:39:33.598662 25723 leader_election.cc:302] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed1853bf1c249a993ebc0e5da8161eb, 313df5efe7ed4ba7ae0d4f1981d3322d; no voters:
I20260926 20:39:33.599002 26094 raft_consensus.cc:2781] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:33.599346 25929 raft_consensus.cc:2454] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 313df5efe7ed4ba7ae0d4f1981d3322d in term 1.
I20260926 20:39:33.599570 26094 raft_consensus.cc:686] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 LEADER]: Becoming Leader. State: Replica: 313df5efe7ed4ba7ae0d4f1981d3322d, State: Running, Role: LEADER
I20260926 20:39:33.600045 26094 consensus_queue.cc:237] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.604265 25491 catalog_manager.cc:5495] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d reported cstate change: term changed from 0 to 1, leader changed from <none> to 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194). New cstate: current_term: 1 leader_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } health_report { overall_health: UNKNOWN } } }
I20260926 20:39:33.694895 26102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:33.687699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46594 (local address 127.23.44.194:41997)
0926 20:39:33.688785 (+ 1086us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:33.688792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:33.688820 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:39:33.688932 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:33.688936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:33.689017 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:33.689206 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:39:33.689703 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.690714 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.692103 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.692740 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.692900 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:33.693397 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:39:33.694146 (+ 749us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:39:33.694346 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:33.694399 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 20:39:33.694485 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":298,"threads_started":1}
I20260926 20:39:33.712033 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:33.705843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:58463 (local address 127.23.44.193:44155)
0926 20:39:33.706006 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:33.706011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:33.706540 (+ 529us) server_negotiation.cc:407] Connection header received
0926 20:39:33.706703 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:33.706706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:33.706775 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:33.706871 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:39:33.707327 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.708039 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.709805 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.710487 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.710666 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:33.711217 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:39:33.711332 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:39:33.711432 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:33.711695 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 20:39:33.711776 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 20:39:33.712181 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:33.705735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44155 (local address 127.23.44.194:58463)
0926 20:39:33.706415 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:33.706437 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:33.706585 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:33.706887 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:33.706890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:33.706900 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:39:33.707180 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:33.707189 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.708154 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.708158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:33.709634 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:33.709647 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.710746 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.710751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:33.710918 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:33.710981 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:39:33.711506 (+ 525us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:39:33.711523 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 20:39:33.711625 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 20:39:33.711797 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":330,"threads_started":1}
I20260926 20:39:33.721167 25929 raft_consensus.cc:1261] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Refusing update from remote peer 313df5efe7ed4ba7ae0d4f1981d3322d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:33.721365 25655 raft_consensus.cc:1261] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Refusing update from remote peer 313df5efe7ed4ba7ae0d4f1981d3322d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:33.721834 26099 consensus_queue.cc:1035] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [LEADER]: Connected to new peer: Peer: permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:33.722188 26094 consensus_queue.cc:1035] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:33.726778 26060 mvcc.cc:205] Tried to move back new op lower bound from 7333704391554379776 to 7333704391070601216. Current Snapshot: MvccSnapshot[applied={T|T < 7333704391554379776}]
I20260926 20:39:33.752943 26102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:33.745887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:54533 (local address 127.23.44.194:41997)
0926 20:39:33.746062 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:33.746068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:33.746518 (+ 450us) server_negotiation.cc:407] Connection header received
0926 20:39:33.746664 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:33.746668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:33.746744 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:33.746827 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:39:33.747386 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.748376 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.750640 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.751353 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.751475 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:33.752040 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:39:33.752152 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:39:33.752288 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:33.752589 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 20:39:33.752689 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 20:39:33.753108 26110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:33.745683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:41997 (local address 127.23.44.193:54533)
0926 20:39:33.746414 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:33.746436 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:33.746554 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:33.746882 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:33.746886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:33.746897 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:39:33.747236 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:33.747245 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.748686 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.748697 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:33.750471 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:33.750488 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.751572 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.751578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:33.751743 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:33.751794 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:39:33.752351 (+ 557us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:39:33.752372 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 20:39:33.752504 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 20:39:33.752746 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":399,"threads_started":1}
I20260926 20:39:33.789796 25492 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:33.790159 25492 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:33.794101 25492 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81: Sending DeleteTablet for 3 replicas of tablet 1fe6bccdba94474289bcdaf78dc580a4
I20260926 20:39:33.795063 25629 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe6bccdba94474289bcdaf78dc580a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:33 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:51634
I20260926 20:39:33.795240 25909 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe6bccdba94474289bcdaf78dc580a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:33 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:57494
I20260926 20:39:33.795476 25773 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe6bccdba94474289bcdaf78dc580a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:33 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:46578
I20260926 20:39:33.797011 25492 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:39:33.797726 25492 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:39:33.805599 26114 tablet_replica.cc:330] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb: stopping tablet replica
I20260926 20:39:33.805920 26114 raft_consensus.cc:2229] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:33.806218 26114 raft_consensus.cc:2258] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:33.806773 26115 tablet_replica.cc:330] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d: stopping tablet replica
I20260926 20:39:33.806993 26115 raft_consensus.cc:2229] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:33.807241 26115 raft_consensus.cc:2258] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:33.807574 26114 ts_tablet_manager.cc:1905] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:33.808416 26115 ts_tablet_manager.cc:1905] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:33.809934 25908 tablet_service.cc:1467] Processing CreateTablet for tablet 850a7665629e462e8722ef3ecab13f99 (DEFAULT_TABLE table=test-table [id=de70519c75c849b18cae1904dd68a32b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:33.810364 25908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850a7665629e462e8722ef3ecab13f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:33.813020 25629 tablet_service.cc:1467] Processing CreateTablet for tablet 850a7665629e462e8722ef3ecab13f99 (DEFAULT_TABLE table=test-table [id=de70519c75c849b18cae1904dd68a32b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:33.813478 25629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850a7665629e462e8722ef3ecab13f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:33.816418 26114 ts_tablet_manager.cc:1918] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:39:33.816460 26115 ts_tablet_manager.cc:1918] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:39:33.816592 26115 log.cc:1196] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/1fe6bccdba94474289bcdaf78dc580a4
I20260926 20:39:33.817155 26115 ts_tablet_manager.cc:1939] T 1fe6bccdba94474289bcdaf78dc580a4 P 62a55f9c956649a08162025051186e3d: Deleting consensus metadata
I20260926 20:39:33.817291 25772 tablet_service.cc:1467] Processing CreateTablet for tablet 850a7665629e462e8722ef3ecab13f99 (DEFAULT_TABLE table=test-table [id=de70519c75c849b18cae1904dd68a32b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:33.817514 26116 tablet_replica.cc:330] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d: stopping tablet replica
I20260926 20:39:33.817692 26114 log.cc:1196] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/1fe6bccdba94474289bcdaf78dc580a4
I20260926 20:39:33.818187 26114 ts_tablet_manager.cc:1939] T 1fe6bccdba94474289bcdaf78dc580a4 P 0ed1853bf1c249a993ebc0e5da8161eb: Deleting consensus metadata
I20260926 20:39:33.817687 25772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850a7665629e462e8722ef3ecab13f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:33.818518 25480 catalog_manager.cc:4841] TS 62a55f9c956649a08162025051186e3d (127.23.44.195:36835): tablet 1fe6bccdba94474289bcdaf78dc580a4 (table test-table [id=41849beed52549a98430140aaa03fa06]) successfully deleted
I20260926 20:39:33.819195 26116 raft_consensus.cc:2229] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:39:33.819891 26116 raft_consensus.cc:2258] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:33.821058 26116 ts_tablet_manager.cc:1905] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:33.826560 25480 catalog_manager.cc:4841] TS 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193:44155): tablet 1fe6bccdba94474289bcdaf78dc580a4 (table test-table [id=41849beed52549a98430140aaa03fa06]) successfully deleted
I20260926 20:39:33.834609 26119 tablet_bootstrap.cc:492] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d: Bootstrap starting.
I20260926 20:39:33.835705 26117 tablet_bootstrap.cc:492] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d: Bootstrap starting.
I20260926 20:39:33.838901 26119 tablet_bootstrap.cc:654] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:33.840412 26118 tablet_bootstrap.cc:492] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb: Bootstrap starting.
I20260926 20:39:33.841112 26117 tablet_bootstrap.cc:654] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:33.844238 26118 tablet_bootstrap.cc:654] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:33.840128 26116 ts_tablet_manager.cc:1918] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:39:33.845727 26116 log.cc:1196] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/1fe6bccdba94474289bcdaf78dc580a4
I20260926 20:39:33.846396 26116 ts_tablet_manager.cc:1939] T 1fe6bccdba94474289bcdaf78dc580a4 P 313df5efe7ed4ba7ae0d4f1981d3322d: Deleting consensus metadata
I20260926 20:39:33.848186 25480 catalog_manager.cc:4841] TS 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194:41997): tablet 1fe6bccdba94474289bcdaf78dc580a4 (table test-table [id=41849beed52549a98430140aaa03fa06]) successfully deleted
I20260926 20:39:33.848513 26119 tablet_bootstrap.cc:492] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d: No bootstrap required, opened a new log
I20260926 20:39:33.848730 26119 ts_tablet_manager.cc:1397] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.002s
I20260926 20:39:33.849525 26119 raft_consensus.cc:348] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.849848 26119 raft_consensus.cc:374] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:33.850039 26119 raft_consensus.cc:729] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 313df5efe7ed4ba7ae0d4f1981d3322d, State: Initialized, Role: FOLLOWER
I20260926 20:39:33.850387 26119 consensus_queue.cc:260] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.851953 26119 ts_tablet_manager.cc:1428] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 20:39:33.856014 26117 tablet_bootstrap.cc:492] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d: No bootstrap required, opened a new log
I20260926 20:39:33.856215 26117 ts_tablet_manager.cc:1397] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.003s
I20260926 20:39:33.856786 26118 tablet_bootstrap.cc:492] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb: No bootstrap required, opened a new log
I20260926 20:39:33.856956 26118 ts_tablet_manager.cc:1397] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.004s
I20260926 20:39:33.857079 26117 raft_consensus.cc:348] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.857316 26117 raft_consensus.cc:374] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:33.857483 26117 raft_consensus.cc:729] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a55f9c956649a08162025051186e3d, State: Initialized, Role: FOLLOWER
I20260926 20:39:33.857740 26117 consensus_queue.cc:260] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.857784 26118 raft_consensus.cc:348] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.858085 26118 raft_consensus.cc:374] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:33.858212 26118 raft_consensus.cc:729] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ed1853bf1c249a993ebc0e5da8161eb, State: Initialized, Role: FOLLOWER
I20260926 20:39:33.858372 26117 ts_tablet_manager.cc:1428] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 20:39:33.858487 26118 consensus_queue.cc:260] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.859354 26118 ts_tablet_manager.cc:1428] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 20:39:33.867038 26067 raft_consensus.cc:482] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:39:33.867255 26067 raft_consensus.cc:504] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.867972 26067 leader_election.cc:288] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193:44155), 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194:41997)
I20260926 20:39:33.868464 25655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850a7665629e462e8722ef3ecab13f99" candidate_uuid: "62a55f9c956649a08162025051186e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" is_pre_election: true
I20260926 20:39:33.868758 25655 raft_consensus.cc:2454] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a55f9c956649a08162025051186e3d in term 0.
I20260926 20:39:33.868747 25793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850a7665629e462e8722ef3ecab13f99" candidate_uuid: "62a55f9c956649a08162025051186e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" is_pre_election: true
I20260926 20:39:33.869014 25793 raft_consensus.cc:2454] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a55f9c956649a08162025051186e3d in term 0.
I20260926 20:39:33.869439 25862 leader_election.cc:302] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed1853bf1c249a993ebc0e5da8161eb, 62a55f9c956649a08162025051186e3d; no voters:
I20260926 20:39:33.869801 26067 raft_consensus.cc:2781] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:39:33.869933 26067 raft_consensus.cc:482] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:39:33.870044 26067 raft_consensus.cc:3037] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:33.872049 26067 raft_consensus.cc:504] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.872635 26067 leader_election.cc:288] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [CANDIDATE]: Term 1 election: Requested vote from peers 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193:44155), 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194:41997)
I20260926 20:39:33.873083 25655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850a7665629e462e8722ef3ecab13f99" candidate_uuid: "62a55f9c956649a08162025051186e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed1853bf1c249a993ebc0e5da8161eb"
I20260926 20:39:33.873284 25655 raft_consensus.cc:3037] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:33.873198 25793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850a7665629e462e8722ef3ecab13f99" candidate_uuid: "62a55f9c956649a08162025051186e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d"
I20260926 20:39:33.873420 25793 raft_consensus.cc:3037] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:33.875320 25655 raft_consensus.cc:2454] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a55f9c956649a08162025051186e3d in term 1.
I20260926 20:39:33.875701 25862 leader_election.cc:302] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed1853bf1c249a993ebc0e5da8161eb, 62a55f9c956649a08162025051186e3d; no voters:
I20260926 20:39:33.876384 26067 raft_consensus.cc:2781] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:33.876744 25793 raft_consensus.cc:2454] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a55f9c956649a08162025051186e3d in term 1.
I20260926 20:39:33.876816 26067 raft_consensus.cc:686] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [term 1 LEADER]: Becoming Leader. State: Replica: 62a55f9c956649a08162025051186e3d, State: Running, Role: LEADER
I20260926 20:39:33.877360 26067 consensus_queue.cc:237] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } }
I20260926 20:39:33.880154 25493 catalog_manager.cc:5495] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 62a55f9c956649a08162025051186e3d (127.23.44.195). New cstate: current_term: 1 leader_uuid: "62a55f9c956649a08162025051186e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62a55f9c956649a08162025051186e3d" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 36835 } health_report { overall_health: HEALTHY } } }
I20260926 20:39:33.913918 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:33.908188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57512 (local address 127.23.44.195:36835)
0926 20:39:33.908374 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:33.908382 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:33.908403 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:39:33.908587 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:33.908593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:33.908683 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:33.908780 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:39:33.909149 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.910244 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.911744 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:33.912397 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:33.912530 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:33.913079 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:39:33.913214 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:39:33.913310 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:33.913517 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 20:39:33.913602 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 20:39:33.924795 25793 raft_consensus.cc:1261] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Refusing update from remote peer 62a55f9c956649a08162025051186e3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:33.925060 25655 raft_consensus.cc:1261] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Refusing update from remote peer 62a55f9c956649a08162025051186e3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:33.925521 26123 consensus_queue.cc:1035] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "313df5efe7ed4ba7ae0d4f1981d3322d" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:33.925935 26067 consensus_queue.cc:1035] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ed1853bf1c249a993ebc0e5da8161eb" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 20:39:33.956342 26059 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 20:39:35.551911 25493 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:35.552206 25493 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:35.557446 25493 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c3c6dadd6a9482489e7bfc5b212fa81: Sending DeleteTablet for 3 replicas of tablet 850a7665629e462e8722ef3ecab13f99
I20260926 20:39:35.558398 25629 tablet_service.cc:1514] Processing DeleteTablet for tablet 850a7665629e462e8722ef3ecab13f99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:35 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:51634
I20260926 20:39:35.558475 25772 tablet_service.cc:1514] Processing DeleteTablet for tablet 850a7665629e462e8722ef3ecab13f99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:35 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:46578
I20260926 20:39:35.558552 25908 tablet_service.cc:1514] Processing DeleteTablet for tablet 850a7665629e462e8722ef3ecab13f99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:35 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:57494
I20260926 20:39:35.559110 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 25564
I20260926 20:39:35.569242 26152 tablet_replica.cc:330] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb: stopping tablet replica
I20260926 20:39:35.569517 26152 raft_consensus.cc:2229] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:35.569846 26152 raft_consensus.cc:2258] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:35.568989 26154 tablet_replica.cc:330] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d: stopping tablet replica
I20260926 20:39:35.570938 26152 ts_tablet_manager.cc:1905] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:35.576759 26154 raft_consensus.cc:2229] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:39:35.577255 26153 tablet_replica.cc:330] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d: stopping tablet replica
I20260926 20:39:35.577656 26154 raft_consensus.cc:2258] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:35.577780 26153 raft_consensus.cc:2229] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:35.578524 26152 ts_tablet_manager.cc:1918] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 20:39:35.578553 26153 raft_consensus.cc:2258] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:35.578794 26152 log.cc:1196] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/850a7665629e462e8722ef3ecab13f99
I20260926 20:39:35.579128 26154 ts_tablet_manager.cc:1905] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:35.579322 26152 ts_tablet_manager.cc:1939] T 850a7665629e462e8722ef3ecab13f99 P 0ed1853bf1c249a993ebc0e5da8161eb: Deleting consensus metadata
I20260926 20:39:35.579793 26153 ts_tablet_manager.cc:1905] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:35.580570 25480 catalog_manager.cc:4841] TS 0ed1853bf1c249a993ebc0e5da8161eb (127.23.44.193:44155): tablet 850a7665629e462e8722ef3ecab13f99 (table test-table [id=de70519c75c849b18cae1904dd68a32b]) successfully deleted
I20260926 20:39:35.588729 26154 ts_tablet_manager.cc:1918] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 20:39:35.588804 26153 ts_tablet_manager.cc:1918] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 20:39:35.588979 26154 log.cc:1196] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/850a7665629e462e8722ef3ecab13f99
I20260926 20:39:35.589108 26153 log.cc:1196] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/850a7665629e462e8722ef3ecab13f99
I20260926 20:39:35.589561 26154 ts_tablet_manager.cc:1939] T 850a7665629e462e8722ef3ecab13f99 P 62a55f9c956649a08162025051186e3d: Deleting consensus metadata
I20260926 20:39:35.589720 26153 ts_tablet_manager.cc:1939] T 850a7665629e462e8722ef3ecab13f99 P 313df5efe7ed4ba7ae0d4f1981d3322d: Deleting consensus metadata
I20260926 20:39:35.590916 25480 catalog_manager.cc:4841] TS 62a55f9c956649a08162025051186e3d (127.23.44.195:36835): tablet 850a7665629e462e8722ef3ecab13f99 (table test-table [id=de70519c75c849b18cae1904dd68a32b]) successfully deleted
I20260926 20:39:35.591212 25480 catalog_manager.cc:4841] TS 313df5efe7ed4ba7ae0d4f1981d3322d (127.23.44.194:41997): tablet 850a7665629e462e8722ef3ecab13f99 (table test-table [id=de70519c75c849b18cae1904dd68a32b]) successfully deleted
Sep 26 20:39:35 dist-test-slave-4d5t krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455173, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:39:35.608350 26156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:35.561355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33220 (local address 127.23.44.193:44155)
0926 20:39:35.567865 (+ 6510us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:35.567875 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:35.575575 (+ 7700us) server_negotiation.cc:407] Connection header received
0926 20:39:35.576033 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:35.576042 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:35.576123 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:35.576249 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:35.584029 (+ 7780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:35.585321 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:35.587757 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:35.588471 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:35.588602 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:35.601484 (+ 12882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:35.601501 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:35.601517 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:35.601563 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:35.605053 (+ 3490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:35.605979 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:35.605986 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:35.605989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:35.606131 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:35.606548 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:35.606553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:35.606556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:35.606928 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:35.607262 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:35.607560 (+ 298us) server_negotiation.cc:299] Negotiation successful
0926 20:39:35.607869 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6304,"thread_start_us":6138,"threads_started":1}
W20260926 20:39:35.609174 25695 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33220
W20260926 20:39:35.609826 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 25564: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:39:35.610060 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 25564
I20260926 20:39:35.633960 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 25704
Sep 26 20:39:35 dist-test-slave-4d5t krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455173, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:39:35.651870 26157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:35.634881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41048 (local address 127.23.44.194:41997)
0926 20:39:35.635483 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:35.635488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:35.635505 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:39:35.635560 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:35.635564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:35.635625 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:35.635762 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:35.637133 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:35.637946 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:35.639306 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:35.639747 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:35.639894 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:35.645323 (+ 5429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:35.645345 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:35.645374 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:35.645427 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:35.648471 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:35.649565 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:35.649576 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:35.649578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:35.649860 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:35.650410 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:35.650423 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:35.650426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:35.650869 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:35.651085 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:35.651413 (+ 328us) server_negotiation.cc:299] Negotiation successful
0926 20:39:35.651588 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":336,"threads_started":1}
W20260926 20:39:35.652645 25834 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41048
W20260926 20:39:35.653301 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 25704: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:39:35.653432 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 25704
I20260926 20:39:35.677646 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 25843
Sep 26 20:39:35 dist-test-slave-4d5t krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455173, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:39:35.695765 26158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:35.678874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47464 (local address 127.23.44.195:36835)
0926 20:39:35.679643 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:35.679648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:35.679665 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:39:35.679717 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:35.679720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:35.679778 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:35.679884 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:35.681488 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:35.682582 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:35.683879 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:35.684332 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:35.684439 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:35.689579 (+ 5140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:35.689597 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:35.689613 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:35.689659 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:35.692826 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:35.693556 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:35.693563 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:35.693566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:35.693727 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:35.694233 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:35.694238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:35.694240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:35.694611 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:35.694910 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:35.695198 (+ 288us) server_negotiation.cc:299] Negotiation successful
0926 20:39:35.695432 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":503,"threads_started":1}
W20260926 20:39:35.696434 25974 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47464
W20260926 20:39:35.697247 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 25843: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:39:35.697435 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 25843
I20260926 20:39:35.721040 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 25461
I20260926 20:39:35.722658 25533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:35.845054 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 25461
2026-09-26T20:39:35Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8792 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:39:35 dist-test-slave-4d5t krb5kdc[26162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:39:35 dist-test-slave-4d5t krb5kdc[26162](info): set up 2 sockets
Sep 26 20:39:35 dist-test-slave-4d5t krb5kdc[26162](info): commencing operation
krb5kdc: starting...
W20260926 20:39:36.969486 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:39:37 dist-test-slave-4d5t krb5kdc[26162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:39:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:39:37.172765 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33939
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:33939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:37.511169 26175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:37.511538 26175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:37.511694 26175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:37.520395 26175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:37.520625 26175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:37.520743 26175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:37.520849 26175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:37.520933 26175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:39:37.531749 26175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:33939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:37.533205 26175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:37.534551 26175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:37.546916 26180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:37.547456 26181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:37.547498 26183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:37.547926 26175 server_base.cc:1034] running on GCE node
I20260926 20:39:37.548806 26175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:37.550333 26175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:37.551585 26175 hybrid_clock.cc:648] HybridClock initialized: now 1790455177551530 us; error 31 us; skew 500 ppm
I20260926 20:39:37.551961 26175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:37 dist-test-slave-4d5t krb5kdc[26162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455177, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:37.555496 26175 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:39:37.557417 26175 webserver.cc:460] Webserver started at http://127.23.44.254:40687/ using document root <none> and password file <none>
I20260926 20:39:37.558136 26175 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:37.558243 26175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:37.558564 26175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:37.561671 26175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "8280f5cc583242ada65e92284565b713"
format_stamp: "Formatted at 2026-09-26 20:39:37 on dist-test-slave-4d5t"
server_key: "9e4ba2526f84c214ee078bb2fadccd57"
server_key_iv: "6e5da053fa173eeb88b418a3a1c8d04d"
server_key_version: "encryptionkey@0"
I20260926 20:39:37.562495 26175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "8280f5cc583242ada65e92284565b713"
format_stamp: "Formatted at 2026-09-26 20:39:37 on dist-test-slave-4d5t"
server_key: "9e4ba2526f84c214ee078bb2fadccd57"
server_key_iv: "6e5da053fa173eeb88b418a3a1c8d04d"
server_key_version: "encryptionkey@0"
I20260926 20:39:37.568391 26175 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 20:39:37.572413 26190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:37.574012 26175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:39:37.574210 26175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "8280f5cc583242ada65e92284565b713"
format_stamp: "Formatted at 2026-09-26 20:39:37 on dist-test-slave-4d5t"
server_key: "9e4ba2526f84c214ee078bb2fadccd57"
server_key_iv: "6e5da053fa173eeb88b418a3a1c8d04d"
server_key_version: "encryptionkey@0"
I20260926 20:39:37.574416 26175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:37.595219 26175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:37.599709 26175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:37.600018 26175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:37.622529 26175 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:33939
I20260926 20:39:37.622540 26241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:33939 every 8 connection(s)
I20260926 20:39:37.624439 26175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:37.628705 26242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:37.633455 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 26175
I20260926 20:39:37.633692 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:37.634644 23731 external_mini_cluster.cc:1428] Setting key b461887845aee83ec42da198d0f6e77d
I20260926 20:39:37.640059 26242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713: Bootstrap starting.
I20260926 20:39:37.643915 26242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:37.645095 26242 log.cc:824] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:37.648375 26242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713: No bootstrap required, opened a new log
Sep 26 20:39:37 dist-test-slave-4d5t krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:37.657023 26242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8280f5cc583242ada65e92284565b713" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33939 } }
I20260926 20:39:37.657413 26242 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:37.657516 26242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8280f5cc583242ada65e92284565b713, State: Initialized, Role: FOLLOWER
I20260926 20:39:37.658165 26242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8280f5cc583242ada65e92284565b713" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33939 } }
I20260926 20:39:37.658484 26242 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:37.658635 26242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:37.658833 26242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:37.661691 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:37.637259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39630 (local address 127.23.44.254:33939)
0926 20:39:37.638236 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:37.638248 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:37.638293 (+ 45us) server_negotiation.cc:407] Connection header received
0926 20:39:37.640173 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:37.640198 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:37.640726 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:37.641074 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:37.643139 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:37.645076 (+ 1937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:37.647386 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:37.648206 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:37.648452 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:37.653195 (+ 4743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:37.653204 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:37.653227 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:37.653272 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:37.656774 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:37.657590 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:37.657599 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:37.657610 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:37.657824 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:37.658257 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:37.658263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:37.658266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:37.659065 (+ 799us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:37.659354 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:37.659785 (+ 431us) server_negotiation.cc:299] Negotiation successful
0926 20:39:37.660263 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":469,"threads_started":1}
I20260926 20:39:37.662218 26242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8280f5cc583242ada65e92284565b713" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33939 } }
I20260926 20:39:37.662823 26242 leader_election.cc:302] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8280f5cc583242ada65e92284565b713; no voters:
I20260926 20:39:37.663537 26242 leader_election.cc:288] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:37.663782 26247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:37.665584 26247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [term 1 LEADER]: Becoming Leader. State: Replica: 8280f5cc583242ada65e92284565b713, State: Running, Role: LEADER
I20260926 20:39:37.666175 26242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:39:37.666115 26247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8280f5cc583242ada65e92284565b713" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33939 } }
I20260926 20:39:37.669306 26249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8280f5cc583242ada65e92284565b713. Latest consensus state: current_term: 1 leader_uuid: "8280f5cc583242ada65e92284565b713" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8280f5cc583242ada65e92284565b713" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33939 } } }
I20260926 20:39:37.669740 26249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:37.670195 26248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8280f5cc583242ada65e92284565b713" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8280f5cc583242ada65e92284565b713" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33939 } } }
I20260926 20:39:37.670516 26248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:37.677317 26256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:37.682276 26256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:37.690739 26256 catalog_manager.cc:1347] Generated new cluster ID: 4fc2f42cb51b4b1e93e8845e96f2e59e
I20260926 20:39:37.690856 26256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:37.707027 26256 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:37.709136 26256 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:37.715960 26256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8280f5cc583242ada65e92284565b713: Generated new TSK 0
I20260926 20:39:37.717120 26256 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:39:37.784371 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33939
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:38.137331 26268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:38.137638 26268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:38.137787 26268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:38.145643 26268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:38.145768 26268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:38.145941 26268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:39:38.156186 26268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:38.157585 26268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:38.158932 26268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:38.173036 26276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:38.173035 26273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:38.173362 26274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:38.173960 26268 server_base.cc:1034] running on GCE node
I20260926 20:39:38.174629 26268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:38.175637 26268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:38.176862 26268 hybrid_clock.cc:648] HybridClock initialized: now 1790455178176799 us; error 57 us; skew 500 ppm
I20260926 20:39:38.177197 26268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:38 dist-test-slave-4d5t krb5kdc[26162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455178, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:38.180634 26268 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:39:38.182169 26268 webserver.cc:460] Webserver started at http://127.23.44.193:40817/ using document root <none> and password file <none>
I20260926 20:39:38.182861 26268 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:38.183012 26268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:38.183313 26268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:38.186287 26268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "f951f4c3dcd0490e8dca87bfa7438514"
format_stamp: "Formatted at 2026-09-26 20:39:38 on dist-test-slave-4d5t"
server_key: "4965d54dad4874f59b1c80254bbb1acf"
server_key_iv: "5c3c3e5927c89b86f153423b706f1f8d"
server_key_version: "encryptionkey@0"
I20260926 20:39:38.187093 26268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "f951f4c3dcd0490e8dca87bfa7438514"
format_stamp: "Formatted at 2026-09-26 20:39:38 on dist-test-slave-4d5t"
server_key: "4965d54dad4874f59b1c80254bbb1acf"
server_key_iv: "5c3c3e5927c89b86f153423b706f1f8d"
server_key_version: "encryptionkey@0"
I20260926 20:39:38.192425 26268 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:39:38.196321 26283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:38.197854 26268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:39:38.198051 26268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "f951f4c3dcd0490e8dca87bfa7438514"
format_stamp: "Formatted at 2026-09-26 20:39:38 on dist-test-slave-4d5t"
server_key: "4965d54dad4874f59b1c80254bbb1acf"
server_key_iv: "5c3c3e5927c89b86f153423b706f1f8d"
server_key_version: "encryptionkey@0"
I20260926 20:39:38.198287 26268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:38.218572 26268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:38.223595 26268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:38.223883 26268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:38.224908 26268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:38.227005 26268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:38.227083 26268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:38.227200 26268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:38.227246 26268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:38.263630 26268 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:38913
I20260926 20:39:38.263671 26395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:38913 every 8 connection(s)
I20260926 20:39:38.265059 26268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:39:38.267624 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 26268
I20260926 20:39:38.267863 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:39:38.268623 23731 external_mini_cluster.cc:1428] Setting key 634fff6787625edfb136aa0f619130e5
Sep 26 20:39:38 dist-test-slave-4d5t krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455178, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
I20260926 20:39:38.291440 26398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:38.267823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:50597 (local address 127.23.44.254:33939)
0926 20:39:38.268315 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:38.268322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:38.269690 (+ 1368us) server_negotiation.cc:407] Connection header received
0926 20:39:38.272320 (+ 2630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:38.272326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:38.272423 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:38.272682 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:38.274964 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:38.275818 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:38.277845 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:38.278718 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:38.278918 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:38.283669 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:38.283679 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:38.283687 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:38.283739 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:38.287688 (+ 3949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:38.288638 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:38.288647 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:38.288652 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:38.288819 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:38.289271 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:38.289277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:38.289283 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:38.289684 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:38.289794 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:38.290753 (+ 959us) server_negotiation.cc:299] Negotiation successful
0926 20:39:38.291017 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":234,"threads_started":1}
I20260926 20:39:38.292237 26399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:38.268496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33939 (local address 127.23.44.193:50597)
0926 20:39:38.269509 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:38.269566 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:38.271895 (+ 2329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:38.272956 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:38.272972 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:38.273817 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:38.274708 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:38.274736 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:38.276015 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:38.276023 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:38.277685 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:38.277699 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:38.279003 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:38.279009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:38.279328 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:38.280221 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:38.280256 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:38.283357 (+ 3101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:38.287899 (+ 4542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:38.287908 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:38.287925 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:38.288454 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:38.288939 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:38.288942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:38.288945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:38.289140 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:38.289811 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:38.289818 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:38.290200 (+ 382us) client_negotiation.cc:769] Sending connection context
0926 20:39:38.290473 (+ 273us) client_negotiation.cc:240] Negotiation successful
0926 20:39:38.290903 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":387,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
I20260926 20:39:38.294490 26396 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33939
I20260926 20:39:38.294818 26396 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:38.295662 26396 heartbeater.cc:511] Master 127.23.44.254:33939 requested a full tablet report, sending...
I20260926 20:39:38.298790 26207 ts_manager.cc:194] Registered new tserver with Master: f951f4c3dcd0490e8dca87bfa7438514 (127.23.44.193:38913)
I20260926 20:39:38.300524 26207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:50597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:39:38.318161 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33939
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:38.674160 26402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:38.674468 26402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:38.674636 26402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:38.682564 26402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:38.682688 26402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:38.682883 26402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:39:38.693192 26402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:38.694537 26402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:38.695945 26402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:38.709120 26410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:38.709298 26407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:38.709323 26408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:38.710350 26402 server_base.cc:1034] running on GCE node
I20260926 20:39:38.710937 26402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:38.711844 26402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:38.713052 26402 hybrid_clock.cc:648] HybridClock initialized: now 1790455178713002 us; error 39 us; skew 500 ppm
I20260926 20:39:38.713419 26402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:38 dist-test-slave-4d5t krb5kdc[26162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455178, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:38.716728 26402 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:39:38.718546 26402 webserver.cc:460] Webserver started at http://127.23.44.194:43455/ using document root <none> and password file <none>
I20260926 20:39:38.719255 26402 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:38.719393 26402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:38.719703 26402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:38.722690 26402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "9138ddbf81ef48ef9fd8deddf9fd31f0"
format_stamp: "Formatted at 2026-09-26 20:39:38 on dist-test-slave-4d5t"
server_key: "87adb9ab540632c67e5bf0337cca36e2"
server_key_iv: "92051c3b42087a3bdb0f509bbc678c4b"
server_key_version: "encryptionkey@0"
I20260926 20:39:38.723510 26402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "9138ddbf81ef48ef9fd8deddf9fd31f0"
format_stamp: "Formatted at 2026-09-26 20:39:38 on dist-test-slave-4d5t"
server_key: "87adb9ab540632c67e5bf0337cca36e2"
server_key_iv: "92051c3b42087a3bdb0f509bbc678c4b"
server_key_version: "encryptionkey@0"
I20260926 20:39:38.728892 26402 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 20:39:38.732607 26417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:38.733994 26402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:39:38.734203 26402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "9138ddbf81ef48ef9fd8deddf9fd31f0"
format_stamp: "Formatted at 2026-09-26 20:39:38 on dist-test-slave-4d5t"
server_key: "87adb9ab540632c67e5bf0337cca36e2"
server_key_iv: "92051c3b42087a3bdb0f509bbc678c4b"
server_key_version: "encryptionkey@0"
I20260926 20:39:38.734416 26402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:38.778342 26402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:38.783201 26402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:38.783519 26402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:38.784575 26402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:38.786064 26402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:38.786167 26402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:38.786319 26402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:38.786389 26402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:38.822795 26402 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:44179
I20260926 20:39:38.822981 26529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:44179 every 8 connection(s)
I20260926 20:39:38.824107 26402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:39:38.825440 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 26402
I20260926 20:39:38.825632 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:39:38.826161 23731 external_mini_cluster.cc:1428] Setting key ad8793817e2c18ec5471da1956e01cc8
Sep 26 20:39:38 dist-test-slave-4d5t krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455178, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
I20260926 20:39:38.846364 26532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:38.827283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:37583 (local address 127.23.44.254:33939)
0926 20:39:38.827718 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:38.827723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:38.828802 (+ 1079us) server_negotiation.cc:407] Connection header received
0926 20:39:38.831224 (+ 2422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:38.831233 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:38.831342 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:38.831482 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:38.833296 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:38.834050 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:38.835614 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:38.836035 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:38.836187 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:38.839863 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:38.839868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:38.839871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:38.839901 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:38.842381 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:38.843166 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:38.843174 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:38.843179 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:38.843327 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:38.843842 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:38.843846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:38.843848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:38.844212 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:38.844341 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:38.845544 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0926 20:39:38.845917 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":217,"threads_started":1}
I20260926 20:39:38.847030 26533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:38.827750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33939 (local address 127.23.44.194:37583)
0926 20:39:38.828628 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:38.828691 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:38.830795 (+ 2104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:38.831789 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:38.831800 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:38.832398 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:38.833105 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:38.833123 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:38.834197 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:38.834212 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:38.835459 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:38.835471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:38.836293 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:38.836300 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:38.836523 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:38.837271 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:38.837293 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:38.839629 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:38.842523 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:38.842533 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:38.842550 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:38.843002 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:38.843454 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:38.843457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:38.843460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:38.843645 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:38.844350 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:38.844356 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:38.844816 (+ 460us) client_negotiation.cc:769] Sending connection context
0926 20:39:38.845065 (+ 249us) client_negotiation.cc:240] Negotiation successful
0926 20:39:38.845619 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":391,"threads_started":1}
I20260926 20:39:38.848662 26530 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33939
I20260926 20:39:38.848984 26530 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:38.849825 26530 heartbeater.cc:511] Master 127.23.44.254:33939 requested a full tablet report, sending...
I20260926 20:39:38.851543 26207 ts_manager.cc:194] Registered new tserver with Master: 9138ddbf81ef48ef9fd8deddf9fd31f0 (127.23.44.194:44179)
I20260926 20:39:38.852392 26207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:37583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:39:38.870512 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33939
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:39.209733 26536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:39.210007 26536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:39.210156 26536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:39.217475 26536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:39.217597 26536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:39.217770 26536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:39:39.228394 26536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:39.229781 26536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:39.231144 26536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:39.246742 26541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:39.246784 26544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:39.246829 26542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:39.247638 26536 server_base.cc:1034] running on GCE node
I20260926 20:39:39.248297 26536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:39.249464 26536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:39.250685 26536 hybrid_clock.cc:648] HybridClock initialized: now 1790455179250620 us; error 64 us; skew 500 ppm
I20260926 20:39:39.251061 26536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:39 dist-test-slave-4d5t krb5kdc[26162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455179, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:39.254941 26536 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:39:39.257225 26536 webserver.cc:460] Webserver started at http://127.23.44.195:45585/ using document root <none> and password file <none>
I20260926 20:39:39.258036 26536 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:39.258193 26536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:39.258528 26536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:39.261821 26536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "f6ada31be0224c54a948a07493693336"
format_stamp: "Formatted at 2026-09-26 20:39:39 on dist-test-slave-4d5t"
server_key: "fc663548a3700bc880cbe99c202ab769"
server_key_iv: "95d28e41a8947f1675ff6fd7a0093c27"
server_key_version: "encryptionkey@0"
I20260926 20:39:39.262641 26536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "f6ada31be0224c54a948a07493693336"
format_stamp: "Formatted at 2026-09-26 20:39:39 on dist-test-slave-4d5t"
server_key: "fc663548a3700bc880cbe99c202ab769"
server_key_iv: "95d28e41a8947f1675ff6fd7a0093c27"
server_key_version: "encryptionkey@0"
I20260926 20:39:39.269182 26536 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 20:39:39.273083 26551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:39.274545 26536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:39:39.274735 26536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "f6ada31be0224c54a948a07493693336"
format_stamp: "Formatted at 2026-09-26 20:39:39 on dist-test-slave-4d5t"
server_key: "fc663548a3700bc880cbe99c202ab769"
server_key_iv: "95d28e41a8947f1675ff6fd7a0093c27"
server_key_version: "encryptionkey@0"
I20260926 20:39:39.274947 26536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:39.305112 26396 heartbeater.cc:503] Master 127.23.44.254:33939 was elected leader, sending a full tablet report...
I20260926 20:39:39.305303 26536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:39.310328 26536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:39.310670 26536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:39.311801 26536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:39.313294 26536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:39.313382 26536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:39.313522 26536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:39.313580 26536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:39.347127 26536 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:36501
I20260926 20:39:39.347285 26663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:36501 every 8 connection(s)
I20260926 20:39:39.348392 26536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:39:39.350816 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 26536
I20260926 20:39:39.351014 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:39:39.351547 23731 external_mini_cluster.cc:1428] Setting key d64c1f62895a21e2aae1c3b60a009d43
Sep 26 20:39:39 dist-test-slave-4d5t krb5kdc[26162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455179, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:39.371017 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:39.351475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:51059 (local address 127.23.44.254:33939)
0926 20:39:39.352010 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:39.352016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:39.353395 (+ 1379us) server_negotiation.cc:407] Connection header received
0926 20:39:39.355849 (+ 2454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:39.355854 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:39.355936 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:39.356034 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:39.357675 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:39.358484 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:39.359994 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:39.360418 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:39.360527 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:39.364134 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:39.364139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:39.364142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:39.364182 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:39.367536 (+ 3354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:39.368408 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:39.368412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:39.368414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:39.368535 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:39.369150 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:39.369154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:39.369156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:39.369427 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:39.369562 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:39.370533 (+ 971us) server_negotiation.cc:299] Negotiation successful
0926 20:39:39.370720 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":285,"threads_started":1}
I20260926 20:39:39.371886 26667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:39.352051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33939 (local address 127.23.44.195:51059)
0926 20:39:39.353247 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:39.353297 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:39.355566 (+ 2269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:39.356297 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:39.356305 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:39.356907 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:39.357481 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:39.357509 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:39.358619 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:39.358623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:39.359849 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:39.359861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:39.360704 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:39.360710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:39.360907 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:39.361555 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:39.361580 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:39.363912 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:39.367757 (+ 3845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:39.367770 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:39.367786 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:39.368250 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:39.368719 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:39.368726 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:39.368733 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:39.369000 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:39.369711 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:39.369718 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:39.370081 (+ 363us) client_negotiation.cc:769] Sending connection context
0926 20:39:39.370312 (+ 231us) client_negotiation.cc:240] Negotiation successful
0926 20:39:39.370724 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":695,"threads_started":1}
I20260926 20:39:39.373589 26664 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33939
I20260926 20:39:39.373961 26664 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:39.374776 26664 heartbeater.cc:511] Master 127.23.44.254:33939 requested a full tablet report, sending...
I20260926 20:39:39.376372 26207 ts_manager.cc:194] Registered new tserver with Master: f6ada31be0224c54a948a07493693336 (127.23.44.195:36501)
I20260926 20:39:39.377367 26207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:51059
I20260926 20:39:39.379889 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 20:39:39.411177 26666 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:39:39.408667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39640 (local address 127.23.44.254:33939)
0926 20:39:39.408838 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:39.408842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:39.409361 (+ 519us) server_negotiation.cc:407] Connection header received
0926 20:39:39.409549 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:39.409554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:39.409644 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:39.409741 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:39.410989 (+ 1248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39640: BlockingRecv error: recv got EOF from 127.0.0.1:39640 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 20:39:39.414336 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 26268
W20260926 20:39:39.417081 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 26268: Not authorized: Client connection negotiation failed: client connection to 127.23.44.193:38913: server requires authentication, but client does not have Kerberos credentials available
I20260926 20:39:39.417275 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 26268
I20260926 20:39:39.434178 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 26402
W20260926 20:39:39.437377 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 26402: Not authorized: Client connection negotiation failed: client connection to 127.23.44.194:44179: server requires authentication, but client does not have Kerberos credentials available
I20260926 20:39:39.437726 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 26402
W20260926 20:39:39.437738 26677 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:39:39.435407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34054 (local address 127.23.44.194:44179)
0926 20:39:39.436030 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:39.436038 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:39.436069 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:39:39.436129 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:39.436135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:39.436354 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:39.436498 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:39.437347 (+ 849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34054: BlockingRecv error: recv got EOF from 127.0.0.1:34054 (error 108)
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":294,"threads_started":1}
I20260926 20:39:39.453588 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 26536
W20260926 20:39:39.456848 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 26536: Not authorized: Client connection negotiation failed: client connection to 127.23.44.195:36501: server requires authentication, but client does not have Kerberos credentials available
I20260926 20:39:39.457016 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 26536
I20260926 20:39:39.473368 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 26175
I20260926 20:39:39.474936 26237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:39.579984 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 26175
2026-09-26T20:39:39Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3729 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:39:39 dist-test-slave-4d5t krb5kdc[26682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:39:39 dist-test-slave-4d5t krb5kdc[26682](info): set up 2 sockets
Sep 26 20:39:39 dist-test-slave-4d5t krb5kdc[26682](info): commencing operation
krb5kdc: starting...
W20260926 20:39:40.701176 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.006s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:39:40 dist-test-slave-4d5t krb5kdc[26682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:39:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:39:40.912451 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:40515
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:40515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:41.248343 26695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:41.248796 26695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:41.248970 26695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:41.257102 26695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:41.257277 26695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:41.257390 26695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:41.257495 26695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:41.257571 26695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:39:41.267614 26695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:40515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:40515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:41.269068 26695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:41.270460 26695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:41.284533 26700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:41.284734 26701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:41.284619 26703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:41.286554 26695 server_base.cc:1034] running on GCE node
I20260926 20:39:41.287671 26695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:41.289740 26695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:41.291011 26695 hybrid_clock.cc:648] HybridClock initialized: now 1790455181290920 us; error 75 us; skew 500 ppm
I20260926 20:39:41.291508 26695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:41 dist-test-slave-4d5t krb5kdc[26682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455181, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:41.303238 26695 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:39:41.305737 26695 webserver.cc:460] Webserver started at http://127.23.44.254:42947/ using document root <none> and password file <none>
I20260926 20:39:41.306533 26695 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:41.306648 26695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:41.306945 26695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:41.310657 26695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "74218a53231c440194f7a75e748608fd"
format_stamp: "Formatted at 2026-09-26 20:39:41 on dist-test-slave-4d5t"
server_key: "180b57b34ceb2c4da6e8a64b2424b0d4"
server_key_iv: "6c36673b05833cffa9df6c9aeb3cd821"
server_key_version: "encryptionkey@0"
I20260926 20:39:41.311811 26695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "74218a53231c440194f7a75e748608fd"
format_stamp: "Formatted at 2026-09-26 20:39:41 on dist-test-slave-4d5t"
server_key: "180b57b34ceb2c4da6e8a64b2424b0d4"
server_key_iv: "6c36673b05833cffa9df6c9aeb3cd821"
server_key_version: "encryptionkey@0"
I20260926 20:39:41.317703 26695 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260926 20:39:41.321610 26710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:41.323849 26695 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:39:41.324091 26695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "74218a53231c440194f7a75e748608fd"
format_stamp: "Formatted at 2026-09-26 20:39:41 on dist-test-slave-4d5t"
server_key: "180b57b34ceb2c4da6e8a64b2424b0d4"
server_key_iv: "6c36673b05833cffa9df6c9aeb3cd821"
server_key_version: "encryptionkey@0"
I20260926 20:39:41.324332 26695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:41.345422 26695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:41.350517 26695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:41.350806 26695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:41.371853 26695 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:40515
I20260926 20:39:41.371883 26761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:40515 every 8 connection(s)
I20260926 20:39:41.373431 26695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:41.374416 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 26695
I20260926 20:39:41.374634 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:41.375419 23731 external_mini_cluster.cc:1428] Setting key 32217d9966c106678cc28c610e0e9afe
I20260926 20:39:41.378471 26762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:39:41 dist-test-slave-4d5t krb5kdc[26682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:41.393469 26762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd: Bootstrap starting.
I20260926 20:39:41.397960 26762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:41.399122 26762 log.cc:824] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:41.402592 26762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd: No bootstrap required, opened a new log
I20260926 20:39:41.402825 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:41.378038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56250 (local address 127.23.44.254:40515)
0926 20:39:41.379083 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:41.379098 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:41.379152 (+ 54us) server_negotiation.cc:407] Connection header received
0926 20:39:41.381209 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:41.381237 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:41.381754 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:41.382238 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:41.384254 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:41.385856 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:41.387970 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:41.388816 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:41.388966 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:41.393896 (+ 4930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:41.393908 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:41.393928 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:41.393985 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:41.398030 (+ 4045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:41.398934 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:41.398943 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:41.398951 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:41.399173 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:41.399584 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:41.399588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:41.399591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:41.400287 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:41.400558 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:41.401072 (+ 514us) server_negotiation.cc:299] Negotiation successful
0926 20:39:41.401499 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":489,"threads_started":1}
I20260926 20:39:41.408331 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:39:41.411844 26762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74218a53231c440194f7a75e748608fd" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40515 } }
I20260926 20:39:41.412482 26762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:41.412688 26762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74218a53231c440194f7a75e748608fd, State: Initialized, Role: FOLLOWER
I20260926 20:39:41.413422 26762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74218a53231c440194f7a75e748608fd" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40515 } }
I20260926 20:39:41.413686 26762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:41.413837 26762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:41.414009 26762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:41.416251 26762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74218a53231c440194f7a75e748608fd" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40515 } }
I20260926 20:39:41.416894 26762 leader_election.cc:302] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74218a53231c440194f7a75e748608fd; no voters:
I20260926 20:39:41.417605 26762 leader_election.cc:288] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:41.417737 26768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:41.418937 26768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [term 1 LEADER]: Becoming Leader. State: Replica: 74218a53231c440194f7a75e748608fd, State: Running, Role: LEADER
I20260926 20:39:41.419584 26768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74218a53231c440194f7a75e748608fd" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40515 } }
I20260926 20:39:41.420147 26762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:39:41.422523 26770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74218a53231c440194f7a75e748608fd. Latest consensus state: current_term: 1 leader_uuid: "74218a53231c440194f7a75e748608fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74218a53231c440194f7a75e748608fd" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40515 } } }
I20260926 20:39:41.422677 26769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74218a53231c440194f7a75e748608fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74218a53231c440194f7a75e748608fd" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40515 } } }
I20260926 20:39:41.422994 26770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:41.423061 26769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:41.435408 26774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:41.445365 26774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:41.456677 26788 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 20:39:41.460656 26774 catalog_manager.cc:1347] Generated new cluster ID: 0ef232be23144d12a686ebeab3204e7e
I20260926 20:39:41.460800 26774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:41.478709 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:41.460218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56262 (local address 127.23.44.254:40515)
0926 20:39:41.460440 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:41.460450 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:41.468641 (+ 8191us) server_negotiation.cc:407] Connection header received
0926 20:39:41.468883 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:41.468892 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:41.468984 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:41.469069 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:41.470677 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:41.471859 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:41.473817 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:41.474514 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:41.474663 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:41.475990 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:41.475999 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:41.476008 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:41.476960 (+ 952us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:41.477032 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:41.477072 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:41.477218 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:41.477367 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 20:39:41.478284 (+ 917us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:39:41.480692 26774 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:41.482053 26774 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:41.498989 26774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74218a53231c440194f7a75e748608fd: Generated new TSK 0
I20260926 20:39:41.500262 26774 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:39:41.518227 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 26695
I20260926 20:39:41.520049 26757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:41.621227 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 26695
2026-09-26T20:39:41Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2038 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:39:41 dist-test-slave-4d5t krb5kdc[26796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:39:41 dist-test-slave-4d5t krb5kdc[26796](info): set up 2 sockets
Sep 26 20:39:41 dist-test-slave-4d5t krb5kdc[26796](info): commencing operation
krb5kdc: starting...
W20260926 20:39:42.745404 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:39:42 dist-test-slave-4d5t krb5kdc[26796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:39:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:39:42.969120 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33415
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:33415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:43.310969 26809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:43.311378 26809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:43.311545 26809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:43.319868 26809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:43.320037 26809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:43.320145 26809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:43.320261 26809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:43.320338 26809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:39:43.331926 26809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:33415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:43.333418 26809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:43.334766 26809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:43.346714 26814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:43.346629 26817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:43.346671 26815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:43.348201 26809 server_base.cc:1034] running on GCE node
I20260926 20:39:43.349130 26809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:43.350698 26809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:43.351961 26809 hybrid_clock.cc:648] HybridClock initialized: now 1790455183351915 us; error 43 us; skew 500 ppm
I20260926 20:39:43.352326 26809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:43 dist-test-slave-4d5t krb5kdc[26796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455183, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:43.355509 26809 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:39:43.357174 26809 webserver.cc:460] Webserver started at http://127.23.44.254:43443/ using document root <none> and password file <none>
I20260926 20:39:43.357897 26809 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:43.358048 26809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:43.358379 26809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:43.361652 26809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "54d5f3cdac5a4ebaa4edcd3ec0b3df14"
format_stamp: "Formatted at 2026-09-26 20:39:43 on dist-test-slave-4d5t"
server_key: "7bd21230d01fb43fd29758e4f6975b03"
server_key_iv: "984470950deb780ec7eb8b50953a6af2"
server_key_version: "encryptionkey@0"
I20260926 20:39:43.362488 26809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "54d5f3cdac5a4ebaa4edcd3ec0b3df14"
format_stamp: "Formatted at 2026-09-26 20:39:43 on dist-test-slave-4d5t"
server_key: "7bd21230d01fb43fd29758e4f6975b03"
server_key_iv: "984470950deb780ec7eb8b50953a6af2"
server_key_version: "encryptionkey@0"
I20260926 20:39:43.368536 26809 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 20:39:43.372426 26824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:43.373981 26809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 20:39:43.374198 26809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "54d5f3cdac5a4ebaa4edcd3ec0b3df14"
format_stamp: "Formatted at 2026-09-26 20:39:43 on dist-test-slave-4d5t"
server_key: "7bd21230d01fb43fd29758e4f6975b03"
server_key_iv: "984470950deb780ec7eb8b50953a6af2"
server_key_version: "encryptionkey@0"
I20260926 20:39:43.374423 26809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:43.387286 26809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:43.392078 26809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:43.392414 26809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:43.412523 26809 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:33415
I20260926 20:39:43.412618 26875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:33415 every 8 connection(s)
I20260926 20:39:43.414093 26809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:43.418187 26876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:43.421726 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 26809
I20260926 20:39:43.421990 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:43.422600 23731 external_mini_cluster.cc:1428] Setting key 51f8381afa359e15f8bd72cedcbd7129
I20260926 20:39:43.433018 26876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14: Bootstrap starting.
I20260926 20:39:43.437625 26876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:43.438781 26876 log.cc:824] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14: Log is configured to *not* fsync() on all Append() calls
Sep 26 20:39:43 dist-test-slave-4d5t krb5kdc[26796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:43.442660 26876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14: No bootstrap required, opened a new log
I20260926 20:39:43.451471 26876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d5f3cdac5a4ebaa4edcd3ec0b3df14" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33415 } }
I20260926 20:39:43.451944 26876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:43.452193 26876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54d5f3cdac5a4ebaa4edcd3ec0b3df14, State: Initialized, Role: FOLLOWER
I20260926 20:39:43.453001 26876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d5f3cdac5a4ebaa4edcd3ec0b3df14" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33415 } }
I20260926 20:39:43.453266 26876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:43.453787 26879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:43.424813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52988 (local address 127.23.44.254:33415)
0926 20:39:43.425667 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:43.425684 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:43.425741 (+ 57us) server_negotiation.cc:407] Connection header received
0926 20:39:43.427641 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:43.427687 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:43.428272 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:43.428781 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:43.430193 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:43.432934 (+ 2741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:43.434924 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:43.436158 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:43.436408 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:43.441150 (+ 4742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:43.441160 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:43.441179 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:43.441228 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:43.445389 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:43.446375 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:43.446382 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:43.446389 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:43.446584 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:43.447058 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:43.447062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:43.447065 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:43.447681 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:43.447928 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:43.448437 (+ 509us) server_negotiation.cc:299] Negotiation successful
0926 20:39:43.452365 (+ 3928us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":380,"threads_started":1}
I20260926 20:39:43.453743 26876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:43.454417 26876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:43.456987 26876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d5f3cdac5a4ebaa4edcd3ec0b3df14" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33415 } }
I20260926 20:39:43.457528 26876 leader_election.cc:302] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54d5f3cdac5a4ebaa4edcd3ec0b3df14; no voters:
I20260926 20:39:43.458130 26876 leader_election.cc:288] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:43.458323 26881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:43.459769 26881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [term 1 LEADER]: Becoming Leader. State: Replica: 54d5f3cdac5a4ebaa4edcd3ec0b3df14, State: Running, Role: LEADER
I20260926 20:39:43.460369 26876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:39:43.460290 26881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d5f3cdac5a4ebaa4edcd3ec0b3df14" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33415 } }
I20260926 20:39:43.463689 26882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54d5f3cdac5a4ebaa4edcd3ec0b3df14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d5f3cdac5a4ebaa4edcd3ec0b3df14" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33415 } } }
I20260926 20:39:43.463680 26883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54d5f3cdac5a4ebaa4edcd3ec0b3df14. Latest consensus state: current_term: 1 leader_uuid: "54d5f3cdac5a4ebaa4edcd3ec0b3df14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54d5f3cdac5a4ebaa4edcd3ec0b3df14" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33415 } } }
I20260926 20:39:43.464216 26883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:43.464236 26882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:43.466635 26888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:43.472301 26888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:43.482443 26888 catalog_manager.cc:1347] Generated new cluster ID: be7907af0c73418c9b9c9352931c8723
I20260926 20:39:43.482581 26888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:43.523175 26888 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:43.524732 26888 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:43.542599 26888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54d5f3cdac5a4ebaa4edcd3ec0b3df14: Generated new TSK 0
I20260926 20:39:43.544220 26888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:39:43.605108 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33415
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:43.954598 26902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:43.954965 26902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:43.955148 26902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:43.962934 26902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:43.963127 26902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:43.963306 26902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:39:43.974881 26902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:43.976507 26902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:43.978601 26902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:43.995536 26907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:43.995676 26908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:43.995772 26910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:43.996646 26902 server_base.cc:1034] running on GCE node
I20260926 20:39:43.997345 26902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:43.998466 26902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:43.999711 26902 hybrid_clock.cc:648] HybridClock initialized: now 1790455183999604 us; error 103 us; skew 500 ppm
I20260926 20:39:44.000073 26902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:43 dist-test-slave-4d5t krb5kdc[26796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455183, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:44.003528 26902 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:39:44.005193 26902 webserver.cc:460] Webserver started at http://127.23.44.193:36925/ using document root <none> and password file <none>
I20260926 20:39:44.006129 26902 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:44.006289 26902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:44.006603 26902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:44.010185 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "83ecf482bf344061be6bcb183cb4b1b7"
format_stamp: "Formatted at 2026-09-26 20:39:44 on dist-test-slave-4d5t"
server_key: "456451e5fcaa7c270364b0328755d178"
server_key_iv: "feeb9c2daecf4d49c51b01f83b965706"
server_key_version: "encryptionkey@0"
I20260926 20:39:44.011417 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "83ecf482bf344061be6bcb183cb4b1b7"
format_stamp: "Formatted at 2026-09-26 20:39:44 on dist-test-slave-4d5t"
server_key: "456451e5fcaa7c270364b0328755d178"
server_key_iv: "feeb9c2daecf4d49c51b01f83b965706"
server_key_version: "encryptionkey@0"
I20260926 20:39:44.018921 26902 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260926 20:39:44.023137 26917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:44.024717 26902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:39:44.024912 26902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "83ecf482bf344061be6bcb183cb4b1b7"
format_stamp: "Formatted at 2026-09-26 20:39:44 on dist-test-slave-4d5t"
server_key: "456451e5fcaa7c270364b0328755d178"
server_key_iv: "feeb9c2daecf4d49c51b01f83b965706"
server_key_version: "encryptionkey@0"
I20260926 20:39:44.025133 26902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:44.043147 26902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:44.047724 26902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:44.048038 26902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:44.049069 26902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:44.050683 26902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:44.050792 26902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:44.050951 26902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:44.051023 26902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:44.090070 26902 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:43261
I20260926 20:39:44.090106 27029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:43261 every 8 connection(s)
I20260926 20:39:44.091418 26902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:39:44.097877 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 26902
I20260926 20:39:44.098089 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:39:44.098706 23731 external_mini_cluster.cc:1428] Setting key 6f4e7bcfd680560d294e9a18ad7ffb52
Sep 26 20:39:44 dist-test-slave-4d5t krb5kdc[26796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455183, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:44.115949 27032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:44.094010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:57855 (local address 127.23.44.254:33415)
0926 20:39:44.094536 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:44.094541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:44.095533 (+ 992us) server_negotiation.cc:407] Connection header received
0926 20:39:44.097501 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:44.097507 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:44.097595 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:44.097725 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:44.099420 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:44.100468 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:44.102141 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:44.102556 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:44.102703 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:44.107341 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:44.107351 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:44.107360 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:44.107411 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:44.111584 (+ 4173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:44.112812 (+ 1228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:44.112823 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:44.112829 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:44.113021 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:44.113668 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:44.113675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:44.113682 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:44.114074 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:44.114246 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:44.115128 (+ 882us) server_negotiation.cc:299] Negotiation successful
0926 20:39:44.115413 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":280,"threads_started":1}
I20260926 20:39:44.116704 27033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:44.094617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33415 (local address 127.23.44.193:57855)
0926 20:39:44.095350 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:44.095396 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:44.097172 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:44.097980 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:44.097989 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:44.098602 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:44.099242 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:44.099260 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:44.100667 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:44.100671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:44.101979 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:44.101992 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:44.102814 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:44.102822 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:44.103060 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:44.103798 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:44.103827 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:44.106986 (+ 3159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:44.111884 (+ 4898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:44.111896 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:44.111919 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:44.112532 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:44.113225 (+ 693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:44.113231 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:44.113234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:44.113513 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:44.114264 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:44.114275 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:44.114708 (+ 433us) client_negotiation.cc:769] Sending connection context
0926 20:39:44.114966 (+ 258us) client_negotiation.cc:240] Negotiation successful
0926 20:39:44.115416 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":274,"threads_started":1}
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
I20260926 20:39:44.118913 27030 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33415
I20260926 20:39:44.119237 27030 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:44.120016 27030 heartbeater.cc:511] Master 127.23.44.254:33415 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
I20260926 20:39:44.122617 26841 ts_manager.cc:194] Registered new tserver with Master: 83ecf482bf344061be6bcb183cb4b1b7 (127.23.44.193:43261)
I20260926 20:39:44.124135 26841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:57855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:39:44.147588 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33415
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:44.500361 27036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:44.500705 27036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:44.500911 27036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:44.508731 27036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:44.508859 27036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:44.509049 27036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:39:44.519408 27036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:44.520844 27036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:44.522280 27036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:44.537892 27042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:44.537916 27041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:44.537847 27044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:44.538787 27036 server_base.cc:1034] running on GCE node
I20260926 20:39:44.539523 27036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:44.540699 27036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:44.541967 27036 hybrid_clock.cc:648] HybridClock initialized: now 1790455184541897 us; error 58 us; skew 500 ppm
I20260926 20:39:44.542322 27036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:44 dist-test-slave-4d5t krb5kdc[26796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455184, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:44.545962 27036 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:39:44.547987 27036 webserver.cc:460] Webserver started at http://127.23.44.194:33245/ using document root <none> and password file <none>
I20260926 20:39:44.548941 27036 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:44.549109 27036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:44.549463 27036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:44.552625 27036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "de0a87b0e8154d24bb6689ca1fbca08c"
format_stamp: "Formatted at 2026-09-26 20:39:44 on dist-test-slave-4d5t"
server_key: "c662d601d333a6b23fb777b38bf3b1e5"
server_key_iv: "afd76e6b0cfe17e1584168746ab309d5"
server_key_version: "encryptionkey@0"
I20260926 20:39:44.553511 27036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "de0a87b0e8154d24bb6689ca1fbca08c"
format_stamp: "Formatted at 2026-09-26 20:39:44 on dist-test-slave-4d5t"
server_key: "c662d601d333a6b23fb777b38bf3b1e5"
server_key_iv: "afd76e6b0cfe17e1584168746ab309d5"
server_key_version: "encryptionkey@0"
I20260926 20:39:44.559187 27036 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:39:44.563163 27051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:44.564733 27036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 20:39:44.564942 27036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "de0a87b0e8154d24bb6689ca1fbca08c"
format_stamp: "Formatted at 2026-09-26 20:39:44 on dist-test-slave-4d5t"
server_key: "c662d601d333a6b23fb777b38bf3b1e5"
server_key_iv: "afd76e6b0cfe17e1584168746ab309d5"
server_key_version: "encryptionkey@0"
I20260926 20:39:44.565176 27036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:44.578152 27036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:44.582686 27036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:44.583029 27036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:44.584030 27036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:44.585579 27036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:44.585687 27036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:44.585836 27036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:44.585909 27036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:44.622900 27036 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:46737
I20260926 20:39:44.622947 27163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:46737 every 8 connection(s)
I20260926 20:39:44.624202 27036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:39:44.630384 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 27036
I20260926 20:39:44.630585 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:39:44.631104 23731 external_mini_cluster.cc:1428] Setting key ec48fc2bf9198c98159d5d99a1d99bcf
Sep 26 20:39:44 dist-test-slave-4d5t krb5kdc[26796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455184, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:44.649987 27166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:44.626952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:45929 (local address 127.23.44.254:33415)
0926 20:39:44.627406 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:44.627410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:44.628463 (+ 1053us) server_negotiation.cc:407] Connection header received
0926 20:39:44.630743 (+ 2280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:44.630748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:44.630818 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:44.630917 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:44.633019 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:44.633880 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:44.636679 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:44.637143 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:44.637259 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:44.641589 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:44.641607 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:44.641615 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:44.641664 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:44.645694 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:44.646563 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:44.646572 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:44.646578 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:44.646745 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:44.647363 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:44.647371 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:44.647378 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:44.647776 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:44.647931 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:44.649184 (+ 1253us) server_negotiation.cc:299] Negotiation successful
0926 20:39:44.649463 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":228,"threads_started":1}
I20260926 20:39:44.650676 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:44.627568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33415 (local address 127.23.44.194:45929)
0926 20:39:44.628282 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:44.628320 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:44.630291 (+ 1971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:44.631201 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:44.631215 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:44.631930 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:44.632796 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:44.632818 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:44.634048 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:44.634055 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:44.636422 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:44.636443 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:44.637404 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:44.637413 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:44.637796 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:44.638776 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:44.638812 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:44.641297 (+ 2485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:44.645891 (+ 4594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:44.645899 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:44.645917 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:44.646405 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:44.646884 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:44.646888 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:44.646891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:44.647208 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:44.647950 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:44.647958 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:44.648485 (+ 527us) client_negotiation.cc:769] Sending connection context
0926 20:39:44.648869 (+ 384us) client_negotiation.cc:240] Negotiation successful
0926 20:39:44.649346 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":283,"threads_started":1}
I20260926 20:39:44.652493 27164 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33415
I20260926 20:39:44.652865 27164 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
I20260926 20:39:44.653695 27164 heartbeater.cc:511] Master 127.23.44.254:33415 requested a full tablet report, sending...
I20260926 20:39:44.655524 26841 ts_manager.cc:194] Registered new tserver with Master: de0a87b0e8154d24bb6689ca1fbca08c (127.23.44.194:46737)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
I20260926 20:39:44.657249 26841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:45929
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:39:44.681973 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33415
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:45.016059 27170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:45.016360 27170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:45.016536 27170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:45.024775 27170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:45.024911 27170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:45.025107 27170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:39:45.035719 27170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:45.037133 27170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:45.038450 27170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:45.056246 27175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:45.056290 27178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:45.056640 27176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:45.059213 27170 server_base.cc:1034] running on GCE node
I20260926 20:39:45.059931 27170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:45.061189 27170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:45.062479 27170 hybrid_clock.cc:648] HybridClock initialized: now 1790455185062399 us; error 62 us; skew 500 ppm
I20260926 20:39:45.062922 27170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:45 dist-test-slave-4d5t krb5kdc[26796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455185, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:45.066378 27170 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:39:45.068472 27170 webserver.cc:460] Webserver started at http://127.23.44.195:34031/ using document root <none> and password file <none>
I20260926 20:39:45.069294 27170 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:45.069504 27170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:45.069906 27170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:45.073052 27170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "b04df1016dc043989b8097e8305d1727"
format_stamp: "Formatted at 2026-09-26 20:39:45 on dist-test-slave-4d5t"
server_key: "0f123009472c8300ad54ff2c9fb5ff79"
server_key_iv: "4d899bb4669002c36c63e9b1afd75acb"
server_key_version: "encryptionkey@0"
I20260926 20:39:45.074261 27170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "b04df1016dc043989b8097e8305d1727"
format_stamp: "Formatted at 2026-09-26 20:39:45 on dist-test-slave-4d5t"
server_key: "0f123009472c8300ad54ff2c9fb5ff79"
server_key_iv: "4d899bb4669002c36c63e9b1afd75acb"
server_key_version: "encryptionkey@0"
I20260926 20:39:45.079919 27170 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 20:39:45.084098 27185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:45.085642 27170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:39:45.085845 27170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "b04df1016dc043989b8097e8305d1727"
format_stamp: "Formatted at 2026-09-26 20:39:45 on dist-test-slave-4d5t"
server_key: "0f123009472c8300ad54ff2c9fb5ff79"
server_key_iv: "4d899bb4669002c36c63e9b1afd75acb"
server_key_version: "encryptionkey@0"
I20260926 20:39:45.086059 27170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:45.106344 27170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:45.111768 27170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:45.112180 27170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:45.113771 27170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:45.115496 27170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:45.115594 27170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:45.115773 27170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:45.115849 27170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:45.128105 27030 heartbeater.cc:503] Master 127.23.44.254:33415 was elected leader, sending a full tablet report...
I20260926 20:39:45.154286 27170 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:40763
I20260926 20:39:45.154321 27297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:40763 every 8 connection(s)
I20260926 20:39:45.155610 27170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:39:45.163574 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 27170
I20260926 20:39:45.163776 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:39:45.164247 23731 external_mini_cluster.cc:1428] Setting key 25381a236d06a92a877ed506b59fd553
Sep 26 20:39:45 dist-test-slave-4d5t krb5kdc[26796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455185, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:45.180053 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:45.158467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:52867 (local address 127.23.44.254:33415)
0926 20:39:45.158951 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:45.158956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:45.160341 (+ 1385us) server_negotiation.cc:407] Connection header received
0926 20:39:45.162926 (+ 2585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:45.162933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:45.163044 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:45.163194 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:45.165364 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:45.166601 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:45.168425 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:45.168957 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:45.169111 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:45.173465 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:45.173481 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:45.173488 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:45.173547 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:45.176819 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:45.177639 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:45.177645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:45.177649 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:45.177780 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:45.178242 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:45.178246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:45.178248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:45.178524 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:45.178679 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:45.179404 (+ 725us) server_negotiation.cc:299] Negotiation successful
0926 20:39:45.179625 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":228,"threads_started":1}
I20260926 20:39:45.181241 27301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:45.159047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33415 (local address 127.23.44.195:52867)
0926 20:39:45.160091 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:45.160151 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:45.162538 (+ 2387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:45.163496 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:45.163506 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:45.164229 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:45.165147 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:45.165180 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:45.166787 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:45.166792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:45.168243 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:45.168257 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:45.169223 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:45.169232 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:45.169544 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:45.170379 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:45.170404 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:45.173048 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:45.176991 (+ 3943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:45.177001 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:45.177027 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:45.177501 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:45.177894 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:45.177899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:45.177903 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:45.178093 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:45.178663 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:45.178672 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:45.179036 (+ 364us) client_negotiation.cc:769] Sending connection context
0926 20:39:45.179317 (+ 281us) client_negotiation.cc:240] Negotiation successful
0926 20:39:45.179913 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":424,"threads_started":1}
I20260926 20:39:45.182726 27298 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33415
I20260926 20:39:45.183111 27298 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:45.183902 27298 heartbeater.cc:511] Master 127.23.44.254:33415 requested a full tablet report, sending...
I20260926 20:39:45.185760 26841 ts_manager.cc:194] Registered new tserver with Master: b04df1016dc043989b8097e8305d1727 (127.23.44.195:40763)
I20260926 20:39:45.187143 26841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:52867
I20260926 20:39:45.193781 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 20:39:45 dist-test-slave-4d5t krb5kdc[26796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455185, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 20:39:45 dist-test-slave-4d5t krb5kdc[26796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455185, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:45.248735 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:45.231737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52996 (local address 127.23.44.254:33415)
0926 20:39:45.231912 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:45.231917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:45.232641 (+ 724us) server_negotiation.cc:407] Connection header received
0926 20:39:45.232787 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:45.232791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:45.232874 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:45.232966 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:45.234655 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:45.235541 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:45.236966 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:45.237415 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:45.237529 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:45.241704 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:45.241711 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:45.241715 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:45.241761 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:45.244960 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:45.246189 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:45.246194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:45.246196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:45.246333 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:45.246942 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:45.246949 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:45.246951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:45.247443 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:45.247692 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:45.248122 (+ 430us) server_negotiation.cc:299] Negotiation successful
0926 20:39:45.248355 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260926 20:39:45.249477 26841 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52996
I20260926 20:39:45.252336 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 26902
Sep 26 20:39:45 dist-test-slave-4d5t krb5kdc[26796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455185, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:39:45.273448 27310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:45.253726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45466 (local address 127.23.44.193:43261)
0926 20:39:45.254498 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:45.254506 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:45.254533 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:39:45.254592 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:45.254599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:45.254818 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:45.254944 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:45.256458 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:45.257744 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:45.260339 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:45.261265 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:45.261472 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:45.266930 (+ 5458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:45.266950 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:45.266967 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:45.267018 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:45.270360 (+ 3342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:45.271176 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:45.271186 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:45.271189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:45.271326 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:45.271724 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:45.271730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:45.271734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:45.272116 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:45.272430 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:45.272804 (+ 374us) server_negotiation.cc:299] Negotiation successful
0926 20:39:45.273087 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":436,"threads_started":1}
W20260926 20:39:45.274680 27025 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45466
W20260926 20:39:45.275741 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 26902: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:39:45.275981 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 26902
I20260926 20:39:45.292869 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 27036
Sep 26 20:39:45 dist-test-slave-4d5t krb5kdc[26796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455185, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:39:45.311113 27311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:45.293802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36974 (local address 127.23.44.194:46737)
0926 20:39:45.294363 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:45.294370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:45.294398 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:39:45.294459 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:45.294465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:45.294688 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:45.294817 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:45.296104 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:45.297239 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:45.298991 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:45.299596 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:45.299718 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:45.304747 (+ 5029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:45.304778 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:45.304801 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:45.304862 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:45.307814 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:45.308768 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:45.308781 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:45.308788 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:45.308967 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:45.309365 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:45.309369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:45.309371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:45.309739 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:45.309996 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:45.310380 (+ 384us) server_negotiation.cc:299] Negotiation successful
0926 20:39:45.310590 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":289,"threads_started":1}
W20260926 20:39:45.313207 27159 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36974
W20260926 20:39:45.314574 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 27036: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:39:45.314791 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 27036
I20260926 20:39:45.331071 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 27170
Sep 26 20:39:45 dist-test-slave-4d5t krb5kdc[26796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455185, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:39:45.348099 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:45.332117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52186 (local address 127.23.44.195:40763)
0926 20:39:45.332904 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:45.332911 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:45.332941 (+ 30us) server_negotiation.cc:407] Connection header received
0926 20:39:45.333003 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:45.333010 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:45.333224 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:45.333357 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:45.334720 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:45.335770 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:45.337162 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:45.337729 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:45.337838 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:45.342296 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:45.342309 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:45.342326 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:45.342365 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:45.345358 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:45.346075 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:45.346083 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:45.346086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:45.346206 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:45.346582 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:45.346586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:45.346588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:45.346940 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:45.347167 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:45.347492 (+ 325us) server_negotiation.cc:299] Negotiation successful
0926 20:39:45.347691 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":525,"threads_started":1}
W20260926 20:39:45.349537 27293 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52186
W20260926 20:39:45.350728 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 27170: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:39:45.350920 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 27170
I20260926 20:39:45.367002 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 26809
I20260926 20:39:45.368220 26871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:45.471710 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 26809
2026-09-26T20:39:45Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3853 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:39:45 dist-test-slave-4d5t krb5kdc[27316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:39:45 dist-test-slave-4d5t krb5kdc[27316](info): set up 2 sockets
Sep 26 20:39:45 dist-test-slave-4d5t krb5kdc[27316](info): commencing operation
krb5kdc: starting...
W20260926 20:39:46.601594 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:39:46 dist-test-slave-4d5t krb5kdc[27316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:39:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:39:46.821122 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:45145
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:45145
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:47.153605 27329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:47.153915 27329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:47.154088 27329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:47.161466 27329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:47.161656 27329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:47.161799 27329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:47.161900 27329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:47.161988 27329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:39:47.172103 27329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:45145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:45145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:47.173529 27329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:47.174854 27329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:47.186705 27335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:47.186780 27337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:47.186753 27334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:47.187493 27329 server_base.cc:1034] running on GCE node
I20260926 20:39:47.188341 27329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:47.190014 27329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:47.191282 27329 hybrid_clock.cc:648] HybridClock initialized: now 1790455187191217 us; error 89 us; skew 500 ppm
I20260926 20:39:47.191670 27329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:47 dist-test-slave-4d5t krb5kdc[27316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455187, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:47.200627 27329 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:39:47.202787 27329 webserver.cc:460] Webserver started at http://127.23.44.254:45571/ using document root <none> and password file <none>
I20260926 20:39:47.203552 27329 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:47.203711 27329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:47.204054 27329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:47.207733 27329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "bf87801c05674c42863a7b790581fe33"
format_stamp: "Formatted at 2026-09-26 20:39:47 on dist-test-slave-4d5t"
server_key: "ebbe2aaefe53e199782816254191c2eb"
server_key_iv: "13884cc694f5427159b8e1db72fadc6b"
server_key_version: "encryptionkey@0"
I20260926 20:39:47.208658 27329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "bf87801c05674c42863a7b790581fe33"
format_stamp: "Formatted at 2026-09-26 20:39:47 on dist-test-slave-4d5t"
server_key: "ebbe2aaefe53e199782816254191c2eb"
server_key_iv: "13884cc694f5427159b8e1db72fadc6b"
server_key_version: "encryptionkey@0"
I20260926 20:39:47.214345 27329 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 20:39:47.218439 27344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:47.219909 27329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:39:47.220124 27329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "bf87801c05674c42863a7b790581fe33"
format_stamp: "Formatted at 2026-09-26 20:39:47 on dist-test-slave-4d5t"
server_key: "ebbe2aaefe53e199782816254191c2eb"
server_key_iv: "13884cc694f5427159b8e1db72fadc6b"
server_key_version: "encryptionkey@0"
I20260926 20:39:47.220341 27329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:47.241676 27329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:47.246806 27329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:47.247118 27329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:47.267298 27329 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:45145
I20260926 20:39:47.267323 27395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:45145 every 8 connection(s)
I20260926 20:39:47.269315 27329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:47.273703 27396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:47.274437 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 27329
I20260926 20:39:47.274719 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:47.275431 23731 external_mini_cluster.cc:1428] Setting key c1940084d479cbb352023c0f6bbbe8c1
I20260926 20:39:47.289877 27396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33: Bootstrap starting.
Sep 26 20:39:47 dist-test-slave-4d5t krb5kdc[27316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:47.293752 27396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:47.295017 27396 log.cc:824] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:47.298163 27396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33: No bootstrap required, opened a new log
I20260926 20:39:47.301869 27399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:47.277921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37040 (local address 127.23.44.254:45145)
0926 20:39:47.278910 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:47.278942 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:47.279000 (+ 58us) server_negotiation.cc:407] Connection header received
0926 20:39:47.281192 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:47.281236 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:47.281751 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:47.282160 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:47.284102 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:47.285999 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:47.287885 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:47.288713 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:47.288863 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:47.293566 (+ 4703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:47.293574 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:47.293590 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:47.293630 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:47.297045 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:47.297806 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:47.297813 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:47.297820 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:47.297978 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:47.298360 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:47.298365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:47.298368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:47.299120 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:47.299434 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:47.299814 (+ 380us) server_negotiation.cc:299] Negotiation successful
0926 20:39:47.300308 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":493,"threads_started":1}
I20260926 20:39:47.308112 27396 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf87801c05674c42863a7b790581fe33" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45145 } }
I20260926 20:39:47.308842 27396 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:47.308984 27396 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf87801c05674c42863a7b790581fe33, State: Initialized, Role: FOLLOWER
I20260926 20:39:47.309653 27396 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf87801c05674c42863a7b790581fe33" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45145 } }
I20260926 20:39:47.309885 27396 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:47.310003 27396 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:47.310151 27396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:47.312983 27396 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf87801c05674c42863a7b790581fe33" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45145 } }
I20260926 20:39:47.313546 27396 leader_election.cc:302] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf87801c05674c42863a7b790581fe33; no voters:
I20260926 20:39:47.314240 27396 leader_election.cc:288] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:47.314484 27401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:47.316326 27401 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [term 1 LEADER]: Becoming Leader. State: Replica: bf87801c05674c42863a7b790581fe33, State: Running, Role: LEADER
I20260926 20:39:47.317034 27401 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf87801c05674c42863a7b790581fe33" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45145 } }
I20260926 20:39:47.317515 27396 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:39:47.320708 27402 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf87801c05674c42863a7b790581fe33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf87801c05674c42863a7b790581fe33" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45145 } } }
I20260926 20:39:47.320205 27403 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf87801c05674c42863a7b790581fe33. Latest consensus state: current_term: 1 leader_uuid: "bf87801c05674c42863a7b790581fe33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf87801c05674c42863a7b790581fe33" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45145 } } }
I20260926 20:39:47.321210 27402 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:47.321247 27403 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:47.322973 27411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:47.329165 27411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:47.338588 27411 catalog_manager.cc:1347] Generated new cluster ID: ffcc98e4f2af4437876455860992f901
I20260926 20:39:47.338724 27411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:47.378714 27411 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:47.380105 27411 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:47.394569 27411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf87801c05674c42863a7b790581fe33: Generated new TSK 0
I20260926 20:39:47.395642 27411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:39:47.458658 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45145
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:47.825223 27422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:47.825579 27422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:47.825752 27422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:47.835644 27422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:47.836030 27422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:47.836282 27422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:39:47.848443 27422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45145
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:47.850389 27422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:47.851871 27422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:47.868566 27430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:47.868485 27428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:47.868623 27427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:47.869544 27422 server_base.cc:1034] running on GCE node
I20260926 20:39:47.870278 27422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:47.871474 27422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:47.872706 27422 hybrid_clock.cc:648] HybridClock initialized: now 1790455187872623 us; error 74 us; skew 500 ppm
I20260926 20:39:47.873111 27422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:47 dist-test-slave-4d5t krb5kdc[27316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455187, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:47.876675 27422 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:39:47.878300 27422 webserver.cc:460] Webserver started at http://127.23.44.193:37405/ using document root <none> and password file <none>
I20260926 20:39:47.879042 27422 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:47.879190 27422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:47.879514 27422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:47.882557 27422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "11b038aa97a543da9764f8733b8c1e22"
format_stamp: "Formatted at 2026-09-26 20:39:47 on dist-test-slave-4d5t"
server_key: "487b22ad7cd3cc6c47e770193ffa4928"
server_key_iv: "249dbe1c93da2c9bf0cf84efec325cca"
server_key_version: "encryptionkey@0"
I20260926 20:39:47.883369 27422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "11b038aa97a543da9764f8733b8c1e22"
format_stamp: "Formatted at 2026-09-26 20:39:47 on dist-test-slave-4d5t"
server_key: "487b22ad7cd3cc6c47e770193ffa4928"
server_key_iv: "249dbe1c93da2c9bf0cf84efec325cca"
server_key_version: "encryptionkey@0"
I20260926 20:39:47.888820 27422 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:39:47.893142 27437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:47.894748 27422 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:39:47.894939 27422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "11b038aa97a543da9764f8733b8c1e22"
format_stamp: "Formatted at 2026-09-26 20:39:47 on dist-test-slave-4d5t"
server_key: "487b22ad7cd3cc6c47e770193ffa4928"
server_key_iv: "249dbe1c93da2c9bf0cf84efec325cca"
server_key_version: "encryptionkey@0"
I20260926 20:39:47.895159 27422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:47.919252 27422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:47.924593 27422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:47.924921 27422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:47.926050 27422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:47.927713 27422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:47.927830 27422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:47.928000 27422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:47.928081 27422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:47.964481 27422 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:40359
I20260926 20:39:47.964529 27549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:40359 every 8 connection(s)
I20260926 20:39:47.965916 27422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:39:47.973006 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 27422
I20260926 20:39:47.973253 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:39:47.974053 23731 external_mini_cluster.cc:1428] Setting key 6251088756f9e6466dcd5a3315d06302
Sep 26 20:39:47 dist-test-slave-4d5t krb5kdc[27316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455187, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:47.991840 27552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:47.969316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:45745 (local address 127.23.44.254:45145)
0926 20:39:47.969774 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:47.969779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:47.970828 (+ 1049us) server_negotiation.cc:407] Connection header received
0926 20:39:47.972952 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:47.972956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:47.973030 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:47.973180 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:47.975323 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:47.976424 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:47.979086 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:47.979525 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:47.979640 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:47.985019 (+ 5379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:47.985025 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:47.985029 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:47.985072 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:47.987939 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:47.988939 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:47.988943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:47.988946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:47.989067 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:47.989764 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:47.989769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:47.989772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:47.990240 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:47.990446 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:47.991221 (+ 775us) server_negotiation.cc:299] Negotiation successful
0926 20:39:47.991453 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":233,"threads_started":1}
I20260926 20:39:47.993005 27553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:47.969918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45145 (local address 127.23.44.193:45745)
0926 20:39:47.970649 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:47.970706 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:47.972593 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:47.973479 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:47.973492 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:47.974303 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:47.975095 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:47.975120 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:47.976624 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:47.976629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:47.978913 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:47.978931 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:47.979778 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:47.979788 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:47.980127 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:47.981097 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:47.981133 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:47.984679 (+ 3546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:47.988151 (+ 3472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:47.988163 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:47.988184 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:47.988791 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:47.989183 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:47.989189 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:47.989193 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:47.989558 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:47.990413 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:47.990424 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:47.990818 (+ 394us) client_negotiation.cc:769] Sending connection context
0926 20:39:47.991106 (+ 288us) client_negotiation.cc:240] Negotiation successful
0926 20:39:47.991701 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":288,"threads_started":1}
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
I20260926 20:39:47.994980 27550 heartbeater.cc:347] Connected to a master server at 127.23.44.254:45145
I20260926 20:39:47.995324 27550 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:47.996081 27550 heartbeater.cc:511] Master 127.23.44.254:45145 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
I20260926 20:39:47.998679 27361 ts_manager.cc:194] Registered new tserver with Master: 11b038aa97a543da9764f8733b8c1e22 (127.23.44.193:40359)
I20260926 20:39:48.000840 27361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:45745
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:39:48.021608 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45145
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:48.379848 27556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:48.380168 27556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:48.380271 27556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:48.387699 27556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:48.387820 27556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:48.387935 27556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:39:48.398061 27556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45145
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:48.399551 27556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:48.400870 27556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:48.414414 27562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:48.414508 27564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:48.414458 27561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:48.415446 27556 server_base.cc:1034] running on GCE node
I20260926 20:39:48.416070 27556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:48.417188 27556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:48.418408 27556 hybrid_clock.cc:648] HybridClock initialized: now 1790455188418341 us; error 56 us; skew 500 ppm
I20260926 20:39:48.418744 27556 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:48 dist-test-slave-4d5t krb5kdc[27316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455188, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:48.422158 27556 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:39:48.423925 27556 webserver.cc:460] Webserver started at http://127.23.44.194:39837/ using document root <none> and password file <none>
I20260926 20:39:48.424762 27556 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:48.424904 27556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:48.425241 27556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:48.428356 27556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "afe93a83d04e40c99a5b6e1590e73a61"
format_stamp: "Formatted at 2026-09-26 20:39:48 on dist-test-slave-4d5t"
server_key: "a6907bfe06514ea084cf76d844de7579"
server_key_iv: "a1fb96505928d4179960b9f6d0b4dc3d"
server_key_version: "encryptionkey@0"
I20260926 20:39:48.429284 27556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "afe93a83d04e40c99a5b6e1590e73a61"
format_stamp: "Formatted at 2026-09-26 20:39:48 on dist-test-slave-4d5t"
server_key: "a6907bfe06514ea084cf76d844de7579"
server_key_iv: "a1fb96505928d4179960b9f6d0b4dc3d"
server_key_version: "encryptionkey@0"
I20260926 20:39:48.434795 27556 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:39:48.438766 27571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:48.440436 27556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:39:48.440722 27556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "afe93a83d04e40c99a5b6e1590e73a61"
format_stamp: "Formatted at 2026-09-26 20:39:48 on dist-test-slave-4d5t"
server_key: "a6907bfe06514ea084cf76d844de7579"
server_key_iv: "a1fb96505928d4179960b9f6d0b4dc3d"
server_key_version: "encryptionkey@0"
I20260926 20:39:48.440975 27556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:48.477533 27556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:48.482999 27556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:48.483340 27556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:48.484473 27556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:48.485978 27556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:48.486088 27556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:48.486249 27556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:48.486330 27556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:48.523608 27556 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:45291
I20260926 20:39:48.523655 27683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:45291 every 8 connection(s)
I20260926 20:39:48.525158 27556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:39:48.535611 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 27556
I20260926 20:39:48.535827 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:39:48.536273 23731 external_mini_cluster.cc:1428] Setting key 8cba51d42c7b648aaee55cf26ef45f53
Sep 26 20:39:48 dist-test-slave-4d5t krb5kdc[27316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455188, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:48.547152 27686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:48.527880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:55317 (local address 127.23.44.254:45145)
0926 20:39:48.528399 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:48.528405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:48.529673 (+ 1268us) server_negotiation.cc:407] Connection header received
0926 20:39:48.531690 (+ 2017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:48.531695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:48.531773 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:48.531891 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:48.533681 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:48.534453 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:48.536513 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:48.537082 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:48.537249 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:48.541041 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:48.541046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:48.541050 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:48.541083 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:48.543674 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:48.544593 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:48.544601 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:48.544604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:48.544745 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:48.545235 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:48.545246 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:48.545249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:48.545586 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:48.545766 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:48.546541 (+ 775us) server_negotiation.cc:299] Negotiation successful
0926 20:39:48.546771 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":293,"threads_started":1}
I20260926 20:39:48.548184 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:48.528538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45145 (local address 127.23.44.194:55317)
0926 20:39:48.529482 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:48.529532 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:48.531372 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:48.532191 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:48.532200 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:48.532879 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:48.533480 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:48.533498 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:48.534655 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:48.534659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:48.536329 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:48.536350 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:48.537354 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:48.537361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:48.537612 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:48.538310 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:48.538338 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:48.540795 (+ 2457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:48.543835 (+ 3040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:48.543845 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:48.543864 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:48.544435 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:48.544878 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:48.544883 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:48.544887 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:48.545109 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:48.545729 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:48.545739 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:48.546176 (+ 437us) client_negotiation.cc:769] Sending connection context
0926 20:39:48.546446 (+ 270us) client_negotiation.cc:240] Negotiation successful
0926 20:39:48.546991 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":326,"threads_started":1}
I20260926 20:39:48.549687 27684 heartbeater.cc:347] Connected to a master server at 127.23.44.254:45145
I20260926 20:39:48.549990 27684 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:48.550719 27684 heartbeater.cc:511] Master 127.23.44.254:45145 requested a full tablet report, sending...
I20260926 20:39:48.552322 27361 ts_manager.cc:194] Registered new tserver with Master: afe93a83d04e40c99a5b6e1590e73a61 (127.23.44.194:45291)
I20260926 20:39:48.553582 27361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:55317
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:39:48.583577 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45145
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:48.928881 27690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:48.929216 27690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:48.929314 27690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:48.936527 27690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:48.936672 27690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:48.936786 27690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:39:48.946888 27690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45145
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:48.948194 27690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:48.949597 27690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:48.963233 27695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:48.963814 27696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:48.964501 27698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:48.969316 27690 server_base.cc:1034] running on GCE node
I20260926 20:39:48.969980 27690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:48.971062 27690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:48.972364 27690 hybrid_clock.cc:648] HybridClock initialized: now 1790455188972302 us; error 41 us; skew 500 ppm
I20260926 20:39:48.972810 27690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:48 dist-test-slave-4d5t krb5kdc[27316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455188, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:48.976845 27690 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:39:48.978623 27690 webserver.cc:460] Webserver started at http://127.23.44.195:33607/ using document root <none> and password file <none>
I20260926 20:39:48.979408 27690 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:48.979565 27690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:48.979892 27690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:48.983070 27690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "f9630c50d8e742f18ab95b6b0f1e9638"
format_stamp: "Formatted at 2026-09-26 20:39:48 on dist-test-slave-4d5t"
server_key: "dab4e5bfc7610a633c901fec71126523"
server_key_iv: "d02fee3a87b353e436649cb10d3bce19"
server_key_version: "encryptionkey@0"
I20260926 20:39:48.983932 27690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "f9630c50d8e742f18ab95b6b0f1e9638"
format_stamp: "Formatted at 2026-09-26 20:39:48 on dist-test-slave-4d5t"
server_key: "dab4e5bfc7610a633c901fec71126523"
server_key_iv: "d02fee3a87b353e436649cb10d3bce19"
server_key_version: "encryptionkey@0"
I20260926 20:39:48.990411 27690 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 20:39:48.994577 27705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:48.996202 27690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:39:48.996412 27690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "f9630c50d8e742f18ab95b6b0f1e9638"
format_stamp: "Formatted at 2026-09-26 20:39:48 on dist-test-slave-4d5t"
server_key: "dab4e5bfc7610a633c901fec71126523"
server_key_iv: "d02fee3a87b353e436649cb10d3bce19"
server_key_version: "encryptionkey@0"
I20260926 20:39:48.996706 27690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:49.004055 27550 heartbeater.cc:503] Master 127.23.44.254:45145 was elected leader, sending a full tablet report...
I20260926 20:39:49.012794 27690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:49.017865 27690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:49.018211 27690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:49.019249 27690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:49.020699 27690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:49.020799 27690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:49.020947 27690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:49.021032 27690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:49.067906 27690 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:33681
I20260926 20:39:49.067956 27817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:33681 every 8 connection(s)
I20260926 20:39:49.069707 27690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:39:49.076475 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 27690
I20260926 20:39:49.076717 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:39:49.077262 23731 external_mini_cluster.cc:1428] Setting key f09ecf95ed4b204916ba35c65b384f09
Sep 26 20:39:49 dist-test-slave-4d5t krb5kdc[27316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455188, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:49.091181 27820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:49.072630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:50725 (local address 127.23.44.254:45145)
0926 20:39:49.073145 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:49.073152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:49.074258 (+ 1106us) server_negotiation.cc:407] Connection header received
0926 20:39:49.076045 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:49.076050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:49.076133 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:49.076241 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:49.078281 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.079355 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.081033 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.081428 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.081563 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:49.085531 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:49.085535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:49.085539 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:49.085570 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:49.088013 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:49.088835 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:49.088845 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:49.088848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:49.088963 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:49.089404 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:49.089409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:49.089412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:49.089672 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:49.089853 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:49.090545 (+ 692us) server_negotiation.cc:299] Negotiation successful
0926 20:39:49.090772 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":227,"threads_started":1}
I20260926 20:39:49.092315 27821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:49.073212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45145 (local address 127.23.44.195:50725)
0926 20:39:49.074067 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:49.074114 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:49.075774 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:49.076543 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:49.076586 (+ 43us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:49.077352 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:49.078096 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:49.078115 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.079507 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.079513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:49.080867 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:49.080879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.081652 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.081659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:49.081995 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:49.082854 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:49.082879 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:49.085286 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:49.088174 (+ 2888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:49.088184 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:49.088202 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:49.088710 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:49.089077 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:49.089083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:49.089086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:49.089285 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:49.089820 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:49.089829 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:49.090180 (+ 351us) client_negotiation.cc:769] Sending connection context
0926 20:39:49.090446 (+ 266us) client_negotiation.cc:240] Negotiation successful
0926 20:39:49.090978 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":386,"threads_started":1}
I20260926 20:39:49.093724 27818 heartbeater.cc:347] Connected to a master server at 127.23.44.254:45145
I20260926 20:39:49.094015 27818 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:49.094704 27818 heartbeater.cc:511] Master 127.23.44.254:45145 requested a full tablet report, sending...
I20260926 20:39:49.096091 27361 ts_manager.cc:194] Registered new tserver with Master: f9630c50d8e742f18ab95b6b0f1e9638 (127.23.44.195:33681)
I20260926 20:39:49.096962 27361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:50725
I20260926 20:39:49.105647 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 20:39:49 dist-test-slave-4d5t krb5kdc[27316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 20:39:49 dist-test-slave-4d5t krb5kdc[27316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:39:49.158582 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:49.142095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46464 (local address 127.23.44.193:40359)
0926 20:39:49.142808 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:49.142818 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:49.142850 (+ 32us) server_negotiation.cc:407] Connection header received
0926 20:39:49.142919 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:49.142926 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:49.143143 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:49.143334 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:49.144972 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.146184 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.147412 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.148000 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.148141 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:49.152469 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:49.152480 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:49.152496 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:49.152533 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:49.155489 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:49.156266 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:49.156283 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:49.156286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:49.156416 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:49.156857 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:49.156864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:49.156867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:49.157238 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:49.157579 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:49.157916 (+ 337us) server_negotiation.cc:299] Negotiation successful
0926 20:39:49.158199 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":372,"threads_started":1}
Sep 26 20:39:49 dist-test-slave-4d5t krb5kdc[27316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:49.177157 27820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:49.163013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37048 (local address 127.23.44.254:45145)
0926 20:39:49.163241 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:49.163246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:49.163396 (+ 150us) server_negotiation.cc:407] Connection header received
0926 20:39:49.163485 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:49.163489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:49.163568 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:49.163653 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:49.165321 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.166104 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.167728 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.168131 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.168260 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:49.172048 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:49.172053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:49.172056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:49.172101 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:49.174695 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:49.175311 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:49.175315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:49.175317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:49.175433 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:49.175746 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:49.175749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:49.175751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:49.176063 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:49.176255 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:49.176656 (+ 401us) server_negotiation.cc:299] Negotiation successful
0926 20:39:49.176826 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260926 20:39:49.177753 27361 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37048
I20260926 20:39:49.178923 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 27422
I20260926 20:39:49.192045 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:49.179908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46476 (local address 127.23.44.193:40359)
0926 20:39:49.180124 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:49.180128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:49.180217 (+ 89us) server_negotiation.cc:407] Connection header received
0926 20:39:49.180338 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:49.180343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:49.180442 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:49.180527 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:49.181800 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.182508 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.183789 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.184234 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.184345 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:49.186734 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:49.186739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:49.186743 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:49.186790 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:49.189650 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:49.190334 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:49.190339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:49.190341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:49.190462 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:49.190805 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:49.190808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:49.190810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:49.191051 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:49.191230 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:49.191504 (+ 274us) server_negotiation.cc:299] Negotiation successful
0926 20:39:49.191677 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 20:39:49.192941 27545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:49.278036 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 27422
I20260926 20:39:49.294332 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 27556
Sep 26 20:39:49 dist-test-slave-4d5t krb5kdc[27316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:39:49.310461 27831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:49.295328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52960 (local address 127.23.44.194:45291)
0926 20:39:49.295875 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:49.295882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:49.295908 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:39:49.295968 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:49.295974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:49.296178 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:49.296342 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:49.297581 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.298581 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.299833 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.300389 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.300504 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:49.304548 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:49.304623 (+ 75us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:49.304653 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:49.304709 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:49.307505 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:49.308174 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:49.308182 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:49.308184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:49.308304 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:49.308886 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:49.308894 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:49.308898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:49.309236 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:49.309527 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:49.309853 (+ 326us) server_negotiation.cc:299] Negotiation successful
0926 20:39:49.310112 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":275,"threads_started":1}
I20260926 20:39:49.312048 27679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:49.398910 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 27556
I20260926 20:39:49.416152 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 27690
Sep 26 20:39:49 dist-test-slave-4d5t krb5kdc[27316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:39:49.435372 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:49.417189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49360 (local address 127.23.44.195:33681)
0926 20:39:49.418181 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:49.418190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:49.418233 (+ 43us) server_negotiation.cc:407] Connection header received
0926 20:39:49.418320 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:49.418329 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:49.418688 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:49.418914 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:49.420245 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.421386 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.422708 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:49.423338 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:49.423524 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:49.428725 (+ 5201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:49.428748 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:49.428769 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:49.428829 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:49.432467 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:49.433284 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:49.433292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:49.433295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:49.433429 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:49.433885 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:49.433889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:49.433891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:49.434286 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:49.434538 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:49.434862 (+ 324us) server_negotiation.cc:299] Negotiation successful
0926 20:39:49.435072 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776,"thread_start_us":488,"threads_started":1}
I20260926 20:39:49.436817 27813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:49.520474 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 27690
I20260926 20:39:49.538391 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 27329
I20260926 20:39:49.539688 27391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:49.643239 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 27329
2026-09-26T20:39:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4170 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:39:49 dist-test-slave-4d5t krb5kdc[27838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:39:49 dist-test-slave-4d5t krb5kdc[27838](info): set up 2 sockets
Sep 26 20:39:49 dist-test-slave-4d5t krb5kdc[27838](info): commencing operation
krb5kdc: starting...
W20260926 20:39:50.781563 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:39:50 dist-test-slave-4d5t krb5kdc[27838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:39:50.992214 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:40143
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:42025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:40143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:51.321280 27851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:51.321590 27851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:51.321689 27851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:51.328971 27851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:51.329209 27851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:51.329339 27851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:51.329460 27851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:51.329546 27851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:39:51.339491 27851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:42025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:40143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:40143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:51.340813 27851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:51.342103 27851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:51.355728 27856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:51.355792 27857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:51.355845 27859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:51.361995 27851 server_base.cc:1034] running on GCE node
I20260926 20:39:51.362844 27851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:51.364358 27851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:51.365586 27851 hybrid_clock.cc:648] HybridClock initialized: now 1790455191365535 us; error 87 us; skew 500 ppm
I20260926 20:39:51.365976 27851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:51 dist-test-slave-4d5t krb5kdc[27838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455191, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:51.369247 27851 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:39:51.370759 27851 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:51.370936 27851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:51.371285 27851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:51.374464 27851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "d3a82d6038764d289a095ebe4fc632ea"
format_stamp: "Formatted at 2026-09-26 20:39:51 on dist-test-slave-4d5t"
server_key: "f235fac62631ee845da7f94791eb225c"
server_key_iv: "82fdff3e22ab8a56c8e2fb558529e330"
server_key_version: "encryptionkey@0"
I20260926 20:39:51.375294 27851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "d3a82d6038764d289a095ebe4fc632ea"
format_stamp: "Formatted at 2026-09-26 20:39:51 on dist-test-slave-4d5t"
server_key: "f235fac62631ee845da7f94791eb225c"
server_key_iv: "82fdff3e22ab8a56c8e2fb558529e330"
server_key_version: "encryptionkey@0"
I20260926 20:39:51.381204 27851 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.006s
I20260926 20:39:51.385092 27865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:51.387128 27851 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:39:51.387465 27851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "d3a82d6038764d289a095ebe4fc632ea"
format_stamp: "Formatted at 2026-09-26 20:39:51 on dist-test-slave-4d5t"
server_key: "f235fac62631ee845da7f94791eb225c"
server_key_iv: "82fdff3e22ab8a56c8e2fb558529e330"
server_key_version: "encryptionkey@0"
I20260926 20:39:51.387775 27851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:51.410526 27851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:51.415086 27851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:51.415387 27851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:51.436640 27851 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:40143
I20260926 20:39:51.436739 27916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:40143 every 8 connection(s)
I20260926 20:39:51.437628 27851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:51.441579 27917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:51.442670 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 27851
I20260926 20:39:51.443082 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:51.443758 23731 external_mini_cluster.cc:1428] Setting key d81fd0ec0c1bc4ae778dd36dbbc10876
I20260926 20:39:51.455655 27917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea: Bootstrap starting.
Sep 26 20:39:51 dist-test-slave-4d5t krb5kdc[27838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:51.459664 27917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:51.461604 27917 log.cc:824] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:51.465214 27917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea: No bootstrap required, opened a new log
I20260926 20:39:51.467228 27920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:51.445862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41704 (local address 127.23.44.254:40143)
0926 20:39:51.446729 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:51.446740 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:51.446786 (+ 46us) server_negotiation.cc:407] Connection header received
0926 20:39:51.448333 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:51.448354 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:51.448812 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:51.449149 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:51.450948 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:51.452285 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:51.454217 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:51.454947 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:51.455093 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:51.459885 (+ 4792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:51.459893 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:51.459909 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:51.459949 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:51.463301 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:51.463993 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:51.464000 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:51.464006 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:51.464163 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:51.464532 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:51.464539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:51.464542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:51.465125 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:51.465443 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:51.465876 (+ 433us) server_negotiation.cc:299] Negotiation successful
0926 20:39:51.466290 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":426,"threads_started":1}
I20260926 20:39:51.474193 27917 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a82d6038764d289a095ebe4fc632ea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40143 } }
I20260926 20:39:51.474637 27917 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:51.474766 27917 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3a82d6038764d289a095ebe4fc632ea, State: Initialized, Role: FOLLOWER
I20260926 20:39:51.475333 27917 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a82d6038764d289a095ebe4fc632ea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40143 } }
I20260926 20:39:51.475576 27917 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:51.475721 27917 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:51.475907 27917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:51.478435 27917 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a82d6038764d289a095ebe4fc632ea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40143 } }
I20260926 20:39:51.478992 27917 leader_election.cc:302] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3a82d6038764d289a095ebe4fc632ea; no voters:
I20260926 20:39:51.479758 27917 leader_election.cc:288] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:51.479877 27922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:51.480698 27922 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [term 1 LEADER]: Becoming Leader. State: Replica: d3a82d6038764d289a095ebe4fc632ea, State: Running, Role: LEADER
I20260926 20:39:51.481266 27922 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a82d6038764d289a095ebe4fc632ea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40143 } }
I20260926 20:39:51.482281 27917 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:39:51.484678 27924 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3a82d6038764d289a095ebe4fc632ea. Latest consensus state: current_term: 1 leader_uuid: "d3a82d6038764d289a095ebe4fc632ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a82d6038764d289a095ebe4fc632ea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40143 } } }
I20260926 20:39:51.485107 27924 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:51.484978 27923 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3a82d6038764d289a095ebe4fc632ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a82d6038764d289a095ebe4fc632ea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40143 } } }
I20260926 20:39:51.485327 27923 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:51.487751 27931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:51.492945 27931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:51.501863 27931 catalog_manager.cc:1347] Generated new cluster ID: d85ef1ef6ffd4a408867b4238c26eeef
I20260926 20:39:51.501981 27931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:51.509498 27931 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:51.510777 27931 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:51.519112 27931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea: Generated new TSK 0
I20260926 20:39:51.519976 27931 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:39:51.571602 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:40143
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:42025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:51.923777 27943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:51.924077 27943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:51.924243 27943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:51.931725 27943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:51.931835 27943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:51.932013 27943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:39:51.942461 27943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:42025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:40143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:51.943828 27943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:51.945220 27943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:51.959142 27948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:51.958988 27949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:51.959225 27951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:51.960695 27943 server_base.cc:1034] running on GCE node
I20260926 20:39:51.961401 27943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:51.962482 27943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:51.963755 27943 hybrid_clock.cc:648] HybridClock initialized: now 1790455191963715 us; error 58 us; skew 500 ppm
I20260926 20:39:51.964115 27943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:51 dist-test-slave-4d5t krb5kdc[27838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455191, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:51.967324 27943 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:39:51.968441 27943 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:51.968586 27943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:51.968958 27943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:51.972440 27943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "9b5126c29a4c4916bc0b3b7237065a06"
format_stamp: "Formatted at 2026-09-26 20:39:51 on dist-test-slave-4d5t"
server_key: "cea8d5a4b34956789a096820e313617d"
server_key_iv: "3a5a87a51d1d46996c6dd6f77d2919d7"
server_key_version: "encryptionkey@0"
I20260926 20:39:51.973409 27943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "9b5126c29a4c4916bc0b3b7237065a06"
format_stamp: "Formatted at 2026-09-26 20:39:51 on dist-test-slave-4d5t"
server_key: "cea8d5a4b34956789a096820e313617d"
server_key_iv: "3a5a87a51d1d46996c6dd6f77d2919d7"
server_key_version: "encryptionkey@0"
I20260926 20:39:51.979449 27943 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260926 20:39:51.983615 27957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:51.985193 27943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:39:51.985357 27943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "9b5126c29a4c4916bc0b3b7237065a06"
format_stamp: "Formatted at 2026-09-26 20:39:51 on dist-test-slave-4d5t"
server_key: "cea8d5a4b34956789a096820e313617d"
server_key_iv: "3a5a87a51d1d46996c6dd6f77d2919d7"
server_key_version: "encryptionkey@0"
I20260926 20:39:51.985620 27943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:52.005056 27943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:52.010227 27943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:52.010521 27943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:52.011550 27943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:52.013234 27943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:52.013341 27943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:52.013432 27943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:52.013500 27943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:52.052768 27943 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:40511
I20260926 20:39:52.052857 28069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:40511 every 8 connection(s)
I20260926 20:39:52.053632 27943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:39:52.054056 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 27943
I20260926 20:39:52.054277 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:39:52.055116 23731 external_mini_cluster.cc:1428] Setting key e482ff8e99637c52b023420ac9394b57
Sep 26 20:39:52 dist-test-slave-4d5t krb5kdc[27838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455191, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
I20260926 20:39:52.075120 28072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:52.056512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:51423 (local address 127.23.44.254:40143)
0926 20:39:52.057034 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:52.057041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:52.058306 (+ 1265us) server_negotiation.cc:407] Connection header received
0926 20:39:52.060082 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:52.060088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:52.060168 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:52.060295 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:52.061856 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:52.062568 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:52.064126 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:52.064532 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:52.064727 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:52.068596 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:52.068606 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:52.068613 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:52.068655 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:52.071613 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:52.072385 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:52.072391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:52.072395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:52.072521 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:52.072984 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:52.072990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:52.072993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:52.073270 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:52.073630 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:52.074427 (+ 797us) server_negotiation.cc:299] Negotiation successful
0926 20:39:52.074661 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":246,"threads_started":1}
I20260926 20:39:52.076300 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:52.056973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:40143 (local address 127.23.44.193:51423)
0926 20:39:52.058116 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:52.058168 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:52.059821 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:52.060516 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:52.060524 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:52.061096 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:52.061672 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:52.061688 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:52.062732 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:52.062738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:52.063972 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:52.063983 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:52.064817 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:52.064826 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:52.065060 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:52.065808 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:52.065830 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:52.068308 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:52.071786 (+ 3478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:52.071796 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:52.071814 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:52.072255 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:52.072678 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:52.072683 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:52.072686 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:52.072867 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:52.073622 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:52.073632 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:52.074008 (+ 376us) client_negotiation.cc:769] Sending connection context
0926 20:39:52.074332 (+ 324us) client_negotiation.cc:240] Negotiation successful
0926 20:39:52.074927 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":482,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
I20260926 20:39:52.078143 28070 heartbeater.cc:347] Connected to a master server at 127.23.44.254:40143
I20260926 20:39:52.078481 28070 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:52.079159 28070 heartbeater.cc:511] Master 127.23.44.254:40143 requested a full tablet report, sending...
I20260926 20:39:52.081542 27882 ts_manager.cc:194] Registered new tserver with Master: 9b5126c29a4c4916bc0b3b7237065a06 (127.23.44.193:40511)
I20260926 20:39:52.083125 27882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:51423
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:39:52.104621 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:40143
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:42025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:52.439412 28076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:52.439782 28076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:52.439934 28076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:52.451676 28076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:52.451841 28076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:52.452028 28076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:39:52.463554 28076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:42025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:40143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:52.464917 28076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:52.466149 28076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:52.478981 28081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:52.478933 28082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:52.479022 28084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:52.479978 28076 server_base.cc:1034] running on GCE node
I20260926 20:39:52.480690 28076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:52.481776 28076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:52.482975 28076 hybrid_clock.cc:648] HybridClock initialized: now 1790455192482929 us; error 41 us; skew 500 ppm
I20260926 20:39:52.483330 28076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:52 dist-test-slave-4d5t krb5kdc[27838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455192, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:52.493330 28076 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:39:52.494853 28076 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:52.494994 28076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:52.495383 28076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:52.498801 28076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "3db07b9ed3ca47909a6b03bfd49df40a"
format_stamp: "Formatted at 2026-09-26 20:39:52 on dist-test-slave-4d5t"
server_key: "b0ff68e536c307e6e862380496a789f4"
server_key_iv: "024e9e3ad8fd538833888963d3644634"
server_key_version: "encryptionkey@0"
I20260926 20:39:52.499768 28076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "3db07b9ed3ca47909a6b03bfd49df40a"
format_stamp: "Formatted at 2026-09-26 20:39:52 on dist-test-slave-4d5t"
server_key: "b0ff68e536c307e6e862380496a789f4"
server_key_iv: "024e9e3ad8fd538833888963d3644634"
server_key_version: "encryptionkey@0"
I20260926 20:39:52.505702 28076 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 20:39:52.510161 28090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:52.512090 28076 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:39:52.512331 28076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "3db07b9ed3ca47909a6b03bfd49df40a"
format_stamp: "Formatted at 2026-09-26 20:39:52 on dist-test-slave-4d5t"
server_key: "b0ff68e536c307e6e862380496a789f4"
server_key_iv: "024e9e3ad8fd538833888963d3644634"
server_key_version: "encryptionkey@0"
I20260926 20:39:52.512681 28076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:52.543465 28076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:52.547879 28076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:52.548197 28076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:52.549305 28076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:52.550784 28076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:52.550869 28076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:52.551023 28076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:52.551087 28076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:52.589964 28076 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:45785
I20260926 20:39:52.590108 28202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:45785 every 8 connection(s)
I20260926 20:39:52.590978 28076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:39:52.596215 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 28076
I20260926 20:39:52.596457 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:39:52.596973 23731 external_mini_cluster.cc:1428] Setting key 9ad542cf1ce92dccc248122ebc8da3de
Sep 26 20:39:52 dist-test-slave-4d5t krb5kdc[27838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455192, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:52.615863 28205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:52.593854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:60069 (local address 127.23.44.254:40143)
0926 20:39:52.594375 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:52.594381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:52.595255 (+ 874us) server_negotiation.cc:407] Connection header received
0926 20:39:52.597485 (+ 2230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:52.597491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:52.597594 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:52.597735 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:52.599440 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:52.600211 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:52.602106 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:52.602577 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:52.602736 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:52.607464 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:52.607474 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:52.607483 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:52.607535 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:52.611753 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:52.612817 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:52.612826 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:52.612832 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:52.612998 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:52.613671 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:52.613677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:52.613681 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:52.614096 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:52.614270 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:52.615115 (+ 845us) server_negotiation.cc:299] Negotiation successful
0926 20:39:52.615406 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":298,"threads_started":1}
I20260926 20:39:52.616623 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:52.594331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:40143 (local address 127.23.44.194:60069)
0926 20:39:52.595069 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:52.595119 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:52.597176 (+ 2057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:52.597989 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:52.597999 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:52.598580 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:52.599254 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:52.599274 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:52.600396 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:52.600400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:52.601929 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:52.601943 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:52.602847 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:52.602855 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:52.603106 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:52.604113 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:52.604138 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:52.607097 (+ 2959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:52.611994 (+ 4897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:52.612004 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:52.612021 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:52.612659 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:52.613153 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:52.613157 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:52.613160 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:52.613512 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:52.614261 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:52.614268 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:52.614658 (+ 390us) client_negotiation.cc:769] Sending connection context
0926 20:39:52.614931 (+ 273us) client_negotiation.cc:240] Negotiation successful
0926 20:39:52.615347 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":302,"threads_started":1}
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
I20260926 20:39:52.618366 28203 heartbeater.cc:347] Connected to a master server at 127.23.44.254:40143
I20260926 20:39:52.618688 28203 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:52.619439 28203 heartbeater.cc:511] Master 127.23.44.254:40143 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
I20260926 20:39:52.621271 27882 ts_manager.cc:194] Registered new tserver with Master: 3db07b9ed3ca47909a6b03bfd49df40a (127.23.44.194:45785)
I20260926 20:39:52.622088 27882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:60069
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:39:52.649158 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:40143
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:42025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:39:53.016530 28209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:53.016933 28209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:53.017132 28209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:53.024926 28209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:53.025053 28209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:53.025228 28209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:39:53.035326 28209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:42025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:40143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:53.036705 28209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:53.038059 28209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:53.052614 28214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:53.052860 28217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:53.054438 28209 server_base.cc:1034] running on GCE node
W20260926 20:39:53.056926 28215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:53.057919 28209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:53.059157 28209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:53.060413 28209 hybrid_clock.cc:648] HybridClock initialized: now 1790455193060308 us; error 97 us; skew 500 ppm
I20260926 20:39:53.060849 28209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:39:53 dist-test-slave-4d5t krb5kdc[27838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455193, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:39:53.064634 28209 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:39:53.066028 28209 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:53.066131 28209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:53.066449 28209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:53.069613 28209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "f382faa31c86403e87dbf9a790fd65b5"
format_stamp: "Formatted at 2026-09-26 20:39:53 on dist-test-slave-4d5t"
server_key: "c907554ac766170e2206c1ccd4cc5bfd"
server_key_iv: "5199f1e9cd79dd2f8b32043598f4bb11"
server_key_version: "encryptionkey@0"
I20260926 20:39:53.070447 28209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "f382faa31c86403e87dbf9a790fd65b5"
format_stamp: "Formatted at 2026-09-26 20:39:53 on dist-test-slave-4d5t"
server_key: "c907554ac766170e2206c1ccd4cc5bfd"
server_key_iv: "5199f1e9cd79dd2f8b32043598f4bb11"
server_key_version: "encryptionkey@0"
I20260926 20:39:53.076520 28209 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260926 20:39:53.080912 28223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:53.082775 28209 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260926 20:39:53.082974 28209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "f382faa31c86403e87dbf9a790fd65b5"
format_stamp: "Formatted at 2026-09-26 20:39:53 on dist-test-slave-4d5t"
server_key: "c907554ac766170e2206c1ccd4cc5bfd"
server_key_iv: "5199f1e9cd79dd2f8b32043598f4bb11"
server_key_version: "encryptionkey@0"
I20260926 20:39:53.083228 28209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:53.086369 28070 heartbeater.cc:503] Master 127.23.44.254:40143 was elected leader, sending a full tablet report...
I20260926 20:39:53.101202 28209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:53.105656 28209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:53.105957 28209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:53.106961 28209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:53.108310 28209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:53.108430 28209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:53.108594 28209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:53.108701 28209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260926 20:39:53.145083 28209 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:46597
I20260926 20:39:53.145141 28335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:46597 every 8 connection(s)
I20260926 20:39:53.146001 28209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:39:53.152667 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 28209
I20260926 20:39:53.152930 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:39:53.153478 23731 external_mini_cluster.cc:1428] Setting key e32d7f60ed4c3d24082cebe6fee671d7
Sep 26 20:39:53 dist-test-slave-4d5t krb5kdc[27838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455193, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:39:53.168497 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.149191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:40459 (local address 127.23.44.254:40143)
0926 20:39:53.149815 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:53.149820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:53.150514 (+ 694us) server_negotiation.cc:407] Connection header received
0926 20:39:53.152299 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:53.152305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:53.152402 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:53.152524 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:53.154435 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.155420 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.157945 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.158428 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.158541 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.162332 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:53.162337 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:53.162340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:53.162372 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:53.165358 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:53.166160 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:53.166165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:53.166168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:53.166290 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:53.166678 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:53.166681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:53.166683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:53.166932 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:53.167045 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:53.167856 (+ 811us) server_negotiation.cc:299] Negotiation successful
0926 20:39:53.168114 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":339,"threads_started":1}
I20260926 20:39:53.169646 28339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.149461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:40143 (local address 127.23.44.195:40459)
0926 20:39:53.150295 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:53.150340 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:53.152000 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:53.152827 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:53.152841 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:53.153429 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:53.154213 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:53.154239 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.155657 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.155663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:53.157761 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:53.157781 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.158699 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.158705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:53.158905 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.159601 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:39:53.159625 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:53.162087 (+ 2462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:53.165533 (+ 3446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:53.165541 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:53.165554 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:53.166001 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:53.166396 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:39:53.166400 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:39:53.166402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:39:53.166573 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:39:53.167051 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:53.167057 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:53.167431 (+ 374us) client_negotiation.cc:769] Sending connection context
0926 20:39:53.167746 (+ 315us) client_negotiation.cc:240] Negotiation successful
0926 20:39:53.168263 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":363,"threads_started":1}
I20260926 20:39:53.171114 28336 heartbeater.cc:347] Connected to a master server at 127.23.44.254:40143
I20260926 20:39:53.171448 28336 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:53.172113 28336 heartbeater.cc:511] Master 127.23.44.254:40143 requested a full tablet report, sending...
I20260926 20:39:53.173887 27882 ts_manager.cc:194] Registered new tserver with Master: f382faa31c86403e87dbf9a790fd65b5 (127.23.44.195:46597)
I20260926 20:39:53.174865 27882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:40459
I20260926 20:39:53.182879 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:39:53.201924 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.187804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41720 (local address 127.23.44.254:40143)
0926 20:39:53.188052 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:53.188059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:53.188412 (+ 353us) server_negotiation.cc:407] Connection header received
0926 20:39:53.188660 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:53.188667 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:53.188751 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:53.188871 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:53.190474 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.191235 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.192629 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.193193 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.193349 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.196191 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:53.196196 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:53.196199 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:53.196230 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:53.198923 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:53.199657 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:53.199664 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:53.199668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:53.199819 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:53.200247 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:53.200253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:53.200257 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:53.200674 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:53.200946 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:53.201304 (+ 358us) server_negotiation.cc:299] Negotiation successful
0926 20:39:53.201552 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:39:53.206342 27882 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:39:53.213609 27882 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:39:53.239101 28353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.230093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55570 (local address 127.23.44.194:45785)
0926 20:39:53.231748 (+ 1655us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:53.231756 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:53.231785 (+ 29us) server_negotiation.cc:407] Connection header received
0926 20:39:53.231876 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:53.231885 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:53.232131 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:53.232320 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:53.233187 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.234626 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.236588 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.237979 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.238140 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.238342 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:53.238603 (+ 261us) server_negotiation.cc:299] Negotiation successful
0926 20:39:53.238781 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1491,"thread_start_us":1367,"threads_started":1}
I20260926 20:39:53.239141 28354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.230395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:45785 (local address 127.0.0.1:55570)
0926 20:39:53.231500 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:53.231520 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:53.231646 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:53.232600 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:53.232607 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:53.232632 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:53.233005 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:53.233018 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.234770 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.234775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:53.236399 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:53.236413 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.238174 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.238180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:53.238367 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.238421 (+ 54us) client_negotiation.cc:769] Sending connection context
0926 20:39:53.238563 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 20:39:53.238754 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":22,"thread_start_us":753,"threads_started":1}
I20260926 20:39:53.240243 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.227204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39680 (local address 127.23.44.193:40511)
0926 20:39:53.228281 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:53.228294 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:53.229569 (+ 1275us) server_negotiation.cc:407] Connection header received
0926 20:39:53.229987 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:53.229997 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:53.230251 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:53.230415 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:53.231975 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.233874 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.237080 (+ 3206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.238751 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.238901 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.239165 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:53.239767 (+ 602us) server_negotiation.cc:299] Negotiation successful
0926 20:39:53.239940 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":1121,"threads_started":1}
I20260926 20:39:53.240317 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.226935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:40511 (local address 127.0.0.1:39680)
0926 20:39:53.229420 (+ 2485us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:53.229457 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:53.229736 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:53.230809 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:53.230816 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:53.230870 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:53.231787 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:53.231801 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.234050 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.234055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:53.236865 (+ 2810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:53.236881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.239324 (+ 2443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.239333 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:53.239539 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.239584 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 20:39:53.239694 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 20:39:53.239851 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2230,"mutex_wait_us":355,"spinlock_wait_cycles":2304,"thread_start_us":2073,"threads_started":1}
I20260926 20:39:53.240684 28352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.227990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41522 (local address 127.23.44.195:46597)
0926 20:39:53.229051 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:53.229060 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:53.229675 (+ 615us) server_negotiation.cc:407] Connection header received
0926 20:39:53.230020 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:53.230028 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:53.230294 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:53.230463 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:53.232221 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.234294 (+ 2073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.237226 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.238767 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.238901 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.239501 (+ 600us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:53.240072 (+ 571us) server_negotiation.cc:299] Negotiation successful
0926 20:39:53.240259 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":651,"threads_started":1}
I20260926 20:39:53.241134 28351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.227641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:46597 (local address 127.0.0.1:41522)
0926 20:39:53.229543 (+ 1902us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:53.229566 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:53.229803 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:53.230806 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:53.230811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:53.230836 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:53.231633 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:53.231646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.234444 (+ 2798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.234450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:53.237037 (+ 2587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:53.237052 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.239653 (+ 2601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.239663 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:53.239856 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.239890 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 20:39:53.240655 (+ 765us) client_negotiation.cc:240] Negotiation successful
0926 20:39:53.240811 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1761,"mutex_wait_us":96,"spinlock_wait_cycles":3328,"thread_start_us":2449,"threads_started":1}
I20260926 20:39:53.244541 28138 tablet_service.cc:1467] Processing CreateTablet for tablet a6556bf02c4d408584a7b79e909dade8 (DEFAULT_TABLE table=test-table [id=79db4f35313b442eb759b667d7677373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:53.244702 28271 tablet_service.cc:1467] Processing CreateTablet for tablet a6556bf02c4d408584a7b79e909dade8 (DEFAULT_TABLE table=test-table [id=79db4f35313b442eb759b667d7677373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:53.244696 28005 tablet_service.cc:1467] Processing CreateTablet for tablet a6556bf02c4d408584a7b79e909dade8 (DEFAULT_TABLE table=test-table [id=79db4f35313b442eb759b667d7677373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:53.246042 28138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6556bf02c4d408584a7b79e909dade8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:53.246156 28005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6556bf02c4d408584a7b79e909dade8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:53.246156 28271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6556bf02c4d408584a7b79e909dade8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:53.257632 28355 tablet_bootstrap.cc:492] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a: Bootstrap starting.
I20260926 20:39:53.263161 28356 tablet_bootstrap.cc:492] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06: Bootstrap starting.
I20260926 20:39:53.264441 28355 tablet_bootstrap.cc:654] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:53.266021 28355 log.cc:824] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:53.267436 28356 tablet_bootstrap.cc:654] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:53.269407 28356 log.cc:824] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:53.274014 28358 tablet_bootstrap.cc:492] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5: Bootstrap starting.
I20260926 20:39:53.276458 28355 tablet_bootstrap.cc:492] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a: No bootstrap required, opened a new log
I20260926 20:39:53.276811 28355 ts_tablet_manager.cc:1397] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.008s
I20260926 20:39:53.276856 28356 tablet_bootstrap.cc:492] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06: No bootstrap required, opened a new log
I20260926 20:39:53.277127 28356 ts_tablet_manager.cc:1397] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260926 20:39:53.280100 28358 tablet_bootstrap.cc:654] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:53.281688 28358 log.cc:824] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:53.289116 28356 raft_consensus.cc:348] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5126c29a4c4916bc0b3b7237065a06" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40511 } } peers { permanent_uuid: "3db07b9ed3ca47909a6b03bfd49df40a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45785 } } peers { permanent_uuid: "f382faa31c86403e87dbf9a790fd65b5" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 46597 } }
I20260926 20:39:53.289851 28356 raft_consensus.cc:374] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:53.290031 28356 raft_consensus.cc:729] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b5126c29a4c4916bc0b3b7237065a06, State: Initialized, Role: FOLLOWER
I20260926 20:39:53.290320 28355 raft_consensus.cc:348] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5126c29a4c4916bc0b3b7237065a06" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40511 } } peers { permanent_uuid: "3db07b9ed3ca47909a6b03bfd49df40a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45785 } } peers { permanent_uuid: "f382faa31c86403e87dbf9a790fd65b5" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 46597 } }
I20260926 20:39:53.290921 28355 raft_consensus.cc:374] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:53.291081 28355 raft_consensus.cc:729] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3db07b9ed3ca47909a6b03bfd49df40a, State: Initialized, Role: FOLLOWER
I20260926 20:39:53.290971 28356 consensus_queue.cc:260] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5126c29a4c4916bc0b3b7237065a06" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40511 } } peers { permanent_uuid: "3db07b9ed3ca47909a6b03bfd49df40a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45785 } } peers { permanent_uuid: "f382faa31c86403e87dbf9a790fd65b5" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 46597 } }
I20260926 20:39:53.291889 28355 consensus_queue.cc:260] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5126c29a4c4916bc0b3b7237065a06" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40511 } } peers { permanent_uuid: "3db07b9ed3ca47909a6b03bfd49df40a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45785 } } peers { permanent_uuid: "f382faa31c86403e87dbf9a790fd65b5" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 46597 } }
I20260926 20:39:53.293776 28203 heartbeater.cc:503] Master 127.23.44.254:40143 was elected leader, sending a full tablet report...
I20260926 20:39:53.295826 28355 ts_tablet_manager.cc:1428] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a: Time spent starting tablet: real 0.019s user 0.013s sys 0.002s
I20260926 20:39:53.297549 28356 ts_tablet_manager.cc:1428] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06: Time spent starting tablet: real 0.020s user 0.015s sys 0.001s
I20260926 20:39:53.299826 28358 tablet_bootstrap.cc:492] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5: No bootstrap required, opened a new log
I20260926 20:39:53.300138 28358 ts_tablet_manager.cc:1397] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.006s
W20260926 20:39:53.306094 28071 tablet.cc:2369] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:39:53.308995 28358 raft_consensus.cc:348] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5126c29a4c4916bc0b3b7237065a06" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40511 } } peers { permanent_uuid: "3db07b9ed3ca47909a6b03bfd49df40a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45785 } } peers { permanent_uuid: "f382faa31c86403e87dbf9a790fd65b5" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 46597 } }
I20260926 20:39:53.309515 28358 raft_consensus.cc:374] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:53.309638 28358 raft_consensus.cc:729] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f382faa31c86403e87dbf9a790fd65b5, State: Initialized, Role: FOLLOWER
I20260926 20:39:53.310324 28358 consensus_queue.cc:260] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5126c29a4c4916bc0b3b7237065a06" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40511 } } peers { permanent_uuid: "3db07b9ed3ca47909a6b03bfd49df40a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45785 } } peers { permanent_uuid: "f382faa31c86403e87dbf9a790fd65b5" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 46597 } }
I20260926 20:39:53.311645 28336 heartbeater.cc:503] Master 127.23.44.254:40143 was elected leader, sending a full tablet report...
I20260926 20:39:53.312337 28358 ts_tablet_manager.cc:1428] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
W20260926 20:39:53.343122 28204 tablet.cc:2369] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:39:53.397723 28337 tablet.cc:2369] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:39:53.426421 28361 raft_consensus.cc:482] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:39:53.426748 28361 raft_consensus.cc:504] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5126c29a4c4916bc0b3b7237065a06" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40511 } } peers { permanent_uuid: "3db07b9ed3ca47909a6b03bfd49df40a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45785 } } peers { permanent_uuid: "f382faa31c86403e87dbf9a790fd65b5" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 46597 } }
I20260926 20:39:53.428689 28361 leader_election.cc:288] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3db07b9ed3ca47909a6b03bfd49df40a (127.23.44.194:45785), f382faa31c86403e87dbf9a790fd65b5 (127.23.44.195:46597)
I20260926 20:39:53.435935 28352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.429249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:37357 (local address 127.23.44.195:46597)
0926 20:39:53.429498 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:53.429510 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:53.430112 (+ 602us) server_negotiation.cc:407] Connection header received
0926 20:39:53.430344 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:53.430353 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:53.430412 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:53.430513 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:53.431112 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.431845 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.433886 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.434841 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.435076 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.435173 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:53.435506 (+ 333us) server_negotiation.cc:299] Negotiation successful
0926 20:39:53.435641 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:39:53.436072 28353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.429050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:51393 (local address 127.23.44.194:45785)
0926 20:39:53.429301 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:53.429325 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:53.430274 (+ 949us) server_negotiation.cc:407] Connection header received
0926 20:39:53.430495 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:53.430508 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:53.430570 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:53.430678 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:39:53.431325 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.432071 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.434206 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.435034 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.435227 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.435313 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:53.435663 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 20:39:53.435807 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:39:53.436221 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.429133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:45785 (local address 127.23.44.193:51393)
0926 20:39:53.430158 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:53.430173 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:53.430342 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:53.430692 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:53.430701 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:53.430751 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:53.431160 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:53.431170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.432229 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.432234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:53.434036 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:53.434048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.435292 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.435298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:53.435495 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.435519 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 20:39:53.435634 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 20:39:53.435776 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"mutex_wait_us":76,"spinlock_wait_cycles":768,"thread_start_us":733,"threads_started":1}
I20260926 20:39:53.436656 28364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.429081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:46597 (local address 127.23.44.193:37357)
0926 20:39:53.429978 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:53.429999 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:53.430190 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:53.430529 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:53.430533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:53.430566 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:39:53.430968 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:53.430977 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.431978 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.431985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:53.433732 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:53.433744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.435088 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.435095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:53.435312 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.435362 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 20:39:53.435479 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 20:39:53.435634 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"mutex_wait_us":29,"spinlock_wait_cycles":22016,"thread_start_us":460,"threads_started":1}
I20260926 20:39:53.437800 28158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6556bf02c4d408584a7b79e909dade8" candidate_uuid: "9b5126c29a4c4916bc0b3b7237065a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3db07b9ed3ca47909a6b03bfd49df40a" is_pre_election: true
I20260926 20:39:53.438138 28291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6556bf02c4d408584a7b79e909dade8" candidate_uuid: "9b5126c29a4c4916bc0b3b7237065a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f382faa31c86403e87dbf9a790fd65b5" is_pre_election: true
I20260926 20:39:53.438359 28158 raft_consensus.cc:2454] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b5126c29a4c4916bc0b3b7237065a06 in term 0.
I20260926 20:39:53.438632 28291 raft_consensus.cc:2454] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b5126c29a4c4916bc0b3b7237065a06 in term 0.
I20260926 20:39:53.439251 27960 leader_election.cc:302] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3db07b9ed3ca47909a6b03bfd49df40a, 9b5126c29a4c4916bc0b3b7237065a06, f382faa31c86403e87dbf9a790fd65b5; no voters:
I20260926 20:39:53.439699 28361 raft_consensus.cc:2781] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:39:53.439855 28361 raft_consensus.cc:482] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:39:53.439970 28361 raft_consensus.cc:3037] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:53.442993 28361 raft_consensus.cc:504] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5126c29a4c4916bc0b3b7237065a06" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40511 } } peers { permanent_uuid: "3db07b9ed3ca47909a6b03bfd49df40a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45785 } } peers { permanent_uuid: "f382faa31c86403e87dbf9a790fd65b5" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 46597 } }
I20260926 20:39:53.444571 28361 leader_election.cc:288] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [CANDIDATE]: Term 1 election: Requested vote from peers 3db07b9ed3ca47909a6b03bfd49df40a (127.23.44.194:45785), f382faa31c86403e87dbf9a790fd65b5 (127.23.44.195:46597)
I20260926 20:39:53.444511 28158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6556bf02c4d408584a7b79e909dade8" candidate_uuid: "9b5126c29a4c4916bc0b3b7237065a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3db07b9ed3ca47909a6b03bfd49df40a"
I20260926 20:39:53.444794 28158 raft_consensus.cc:3037] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:53.445154 28291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6556bf02c4d408584a7b79e909dade8" candidate_uuid: "9b5126c29a4c4916bc0b3b7237065a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f382faa31c86403e87dbf9a790fd65b5"
I20260926 20:39:53.445385 28291 raft_consensus.cc:3037] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:53.448078 28158 raft_consensus.cc:2454] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b5126c29a4c4916bc0b3b7237065a06 in term 1.
I20260926 20:39:53.448597 28291 raft_consensus.cc:2454] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b5126c29a4c4916bc0b3b7237065a06 in term 1.
I20260926 20:39:53.448691 27961 leader_election.cc:302] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3db07b9ed3ca47909a6b03bfd49df40a, 9b5126c29a4c4916bc0b3b7237065a06; no voters:
I20260926 20:39:53.449057 28361 raft_consensus.cc:2781] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:53.449551 28361 raft_consensus.cc:686] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [term 1 LEADER]: Becoming Leader. State: Replica: 9b5126c29a4c4916bc0b3b7237065a06, State: Running, Role: LEADER
I20260926 20:39:53.450183 28361 consensus_queue.cc:237] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5126c29a4c4916bc0b3b7237065a06" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40511 } } peers { permanent_uuid: "3db07b9ed3ca47909a6b03bfd49df40a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45785 } } peers { permanent_uuid: "f382faa31c86403e87dbf9a790fd65b5" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 46597 } }
I20260926 20:39:53.454744 27882 catalog_manager.cc:5495] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b5126c29a4c4916bc0b3b7237065a06 (127.23.44.193). New cstate: current_term: 1 leader_uuid: "9b5126c29a4c4916bc0b3b7237065a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b5126c29a4c4916bc0b3b7237065a06" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3db07b9ed3ca47909a6b03bfd49df40a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f382faa31c86403e87dbf9a790fd65b5" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 46597 } health_report { overall_health: UNKNOWN } } }
I20260926 20:39:53.481376 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.476105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39684 (local address 127.23.44.193:40511)
0926 20:39:53.476360 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:53.476366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:53.476385 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:39:53.476629 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:53.476633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:53.476713 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:53.476869 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:39:53.477220 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.477947 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.479266 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.479656 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.479783 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.480236 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:39:53.480817 (+ 581us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:39:53.480979 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:53.481031 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 20:39:53.481119 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 20:39:53.490707 28158 raft_consensus.cc:1261] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a [term 1 FOLLOWER]: Refusing update from remote peer 9b5126c29a4c4916bc0b3b7237065a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:53.490722 28291 raft_consensus.cc:1261] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5 [term 1 FOLLOWER]: Refusing update from remote peer 9b5126c29a4c4916bc0b3b7237065a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:53.491778 28366 consensus_queue.cc:1035] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f382faa31c86403e87dbf9a790fd65b5" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:53.492256 28361 consensus_queue.cc:1035] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3db07b9ed3ca47909a6b03bfd49df40a" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:53.505391 28369 mvcc.cc:205] Tried to move back new op lower bound from 7333704472525586432 to 7333704472376074240. Current Snapshot: MvccSnapshot[applied={T|T < 7333704472525586432}]
I20260926 20:39:53.505963 28371 mvcc.cc:205] Tried to move back new op lower bound from 7333704472525586432 to 7333704472376074240. Current Snapshot: MvccSnapshot[applied={T|T < 7333704472525586432}]
I20260926 20:39:53.521883 27882 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:53.522161 27882 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:53.525949 27882 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3a82d6038764d289a095ebe4fc632ea: Sending DeleteTablet for 3 replicas of tablet a6556bf02c4d408584a7b79e909dade8
I20260926 20:39:53.527402 28005 tablet_service.cc:1514] Processing DeleteTablet for tablet a6556bf02c4d408584a7b79e909dade8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:53 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:39680
I20260926 20:39:53.527390 28271 tablet_service.cc:1514] Processing DeleteTablet for tablet a6556bf02c4d408584a7b79e909dade8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:53 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:41522
I20260926 20:39:53.527588 28138 tablet_service.cc:1514] Processing DeleteTablet for tablet a6556bf02c4d408584a7b79e909dade8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:53 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:55570
I20260926 20:39:53.529497 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 27943
I20260926 20:39:53.540508 28380 tablet_replica.cc:330] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06: stopping tablet replica
I20260926 20:39:53.541182 28380 raft_consensus.cc:2229] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:39:53.542057 28380 raft_consensus.cc:2258] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:53.543926 28381 tablet_replica.cc:330] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a: stopping tablet replica
I20260926 20:39:53.544209 28379 tablet_replica.cc:330] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5: stopping tablet replica
I20260926 20:39:53.544385 28381 raft_consensus.cc:2229] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:53.544673 28379 raft_consensus.cc:2229] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:53.544940 28381 raft_consensus.cc:2258] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:53.545193 28379 raft_consensus.cc:2258] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:53.547222 28381 ts_tablet_manager.cc:1905] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:53.547222 28379 ts_tablet_manager.cc:1905] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 20:39:53 dist-test-slave-4d5t krb5kdc[27838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:39:53.554306 28381 ts_tablet_manager.cc:1918] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:39:53.554489 28381 log.cc:1196] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/a6556bf02c4d408584a7b79e909dade8
I20260926 20:39:53.554615 28379 ts_tablet_manager.cc:1918] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:39:53.554807 28379 log.cc:1196] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/a6556bf02c4d408584a7b79e909dade8
I20260926 20:39:53.555150 28381 ts_tablet_manager.cc:1939] T a6556bf02c4d408584a7b79e909dade8 P 3db07b9ed3ca47909a6b03bfd49df40a: Deleting consensus metadata
I20260926 20:39:53.555380 28379 ts_tablet_manager.cc:1939] T a6556bf02c4d408584a7b79e909dade8 P f382faa31c86403e87dbf9a790fd65b5: Deleting consensus metadata
I20260926 20:39:53.557286 27868 catalog_manager.cc:4841] TS f382faa31c86403e87dbf9a790fd65b5 (127.23.44.195:46597): tablet a6556bf02c4d408584a7b79e909dade8 (table test-table [id=79db4f35313b442eb759b667d7677373]) successfully deleted
I20260926 20:39:53.557996 28380 ts_tablet_manager.cc:1905] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:53.558538 27869 catalog_manager.cc:4841] TS 3db07b9ed3ca47909a6b03bfd49df40a (127.23.44.194:45785): tablet a6556bf02c4d408584a7b79e909dade8 (table test-table [id=79db4f35313b442eb759b667d7677373]) successfully deleted
I20260926 20:39:53.564764 28380 ts_tablet_manager.cc:1918] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:39:53.564989 28380 log.cc:1196] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/a6556bf02c4d408584a7b79e909dade8
I20260926 20:39:53.565584 28380 ts_tablet_manager.cc:1939] T a6556bf02c4d408584a7b79e909dade8 P 9b5126c29a4c4916bc0b3b7237065a06: Deleting consensus metadata
I20260926 20:39:53.566514 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.531338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39688 (local address 127.23.44.193:40511)
0926 20:39:53.531608 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:53.531618 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:53.543180 (+ 11562us) server_negotiation.cc:407] Connection header received
0926 20:39:53.543426 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:53.543432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:53.543519 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:53.543619 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:53.545710 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.546859 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.548757 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.549466 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.549572 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.556704 (+ 7132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:53.556731 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:53.556762 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:53.556856 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:53.562063 (+ 5207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:53.563127 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:53.563143 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:53.563150 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:53.563385 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:53.563828 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:53.563833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:53.563837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:53.564381 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:53.564999 (+ 618us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:53.565285 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 20:39:53.565981 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 20:39:53.567322 28065 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:53.567845 27869 catalog_manager.cc:4841] TS 9b5126c29a4c4916bc0b3b7237065a06 (127.23.44.193:40511): tablet a6556bf02c4d408584a7b79e909dade8 (table test-table [id=79db4f35313b442eb759b667d7677373]) successfully deleted
I20260926 20:39:53.668738 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 27943
I20260926 20:39:53.690135 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 28076
Sep 26 20:39:53 dist-test-slave-4d5t krb5kdc[27838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:39:53.705791 28353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.691048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55574 (local address 127.23.44.194:45785)
0926 20:39:53.691213 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:53.691217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:53.691235 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:39:53.691403 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:53.691407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:53.691484 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:53.691575 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:53.692882 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.693662 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.694977 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.695435 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.695574 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.699776 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:53.699798 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:53.699919 (+ 121us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:53.699967 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:53.702814 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:53.703540 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:53.703547 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:53.703549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:53.703684 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:53.704305 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:53.704310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:53.704312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:53.704806 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:53.705057 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:53.705356 (+ 299us) server_negotiation.cc:299] Negotiation successful
0926 20:39:53.705522 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 20:39:53.706539 28198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:53.798343 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 28076
I20260926 20:39:53.816735 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 28209
Sep 26 20:39:53 dist-test-slave-4d5t krb5kdc[27838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:39:53.834015 28352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:53.817767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41534 (local address 127.23.44.195:46597)
0926 20:39:53.818069 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:53.818074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:53.818097 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:39:53.818180 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:53.818184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:53.818257 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:53.818366 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:53.819888 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.820841 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.822378 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:53.823019 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:53.823143 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:53.827537 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:53.827552 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:53.827569 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:39:53.827608 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:53.830929 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:53.831657 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:53.831665 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:53.831668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:53.831827 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:39:53.832220 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:39:53.832224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:39:53.832226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:39:53.832718 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:53.832947 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:53.833289 (+ 342us) server_negotiation.cc:299] Negotiation successful
0926 20:39:53.833641 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173}
I20260926 20:39:53.834770 28331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:53.927321 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 28209
I20260926 20:39:53.943907 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 27851
I20260926 20:39:53.945230 27912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:54.055373 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 27851
2026-09-26T20:39:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4412 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T20:39:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:54Z Disabled control of system clock
I20260926 20:39:54.125075 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:42797
--webserver_interface=127.23.44.254
--webserver_port=0
--builtin_ntp_servers=127.23.44.212:39395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:42797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 20:39:54.453727 28390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:54.454049 28390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:54.454146 28390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:54.461835 28390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:54.461967 28390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:54.462007 28390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:54.462057 28390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:54.462102 28390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:39:54.471518 28390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:42797
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:42797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:54.472844 28390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:54.474697 28390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:54.487512 28395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:54.487583 28396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:54.487478 28398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:54.488867 28390 server_base.cc:1034] running on GCE node
I20260926 20:39:54.489742 28390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:54.491226 28390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:54.492478 28390 hybrid_clock.cc:648] HybridClock initialized: now 1790455194492418 us; error 53 us; skew 500 ppm
I20260926 20:39:54.492864 28390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:39:54.495757 28390 webserver.cc:460] Webserver started at http://127.23.44.254:37847/ using document root <none> and password file <none>
I20260926 20:39:54.496666 28390 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:54.496803 28390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:54.497145 28390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:54.500222 28390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "080c50841796462985400e3a294a2efc"
format_stamp: "Formatted at 2026-09-26 20:39:54 on dist-test-slave-4d5t"
server_key: "99dc689e53e485ac35ffaf102180a9f3"
server_key_iv: "56f068695b9d198c92f0278cf4c3ddb7"
server_key_version: "encryptionkey@0"
I20260926 20:39:54.501096 28390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "080c50841796462985400e3a294a2efc"
format_stamp: "Formatted at 2026-09-26 20:39:54 on dist-test-slave-4d5t"
server_key: "99dc689e53e485ac35ffaf102180a9f3"
server_key_iv: "56f068695b9d198c92f0278cf4c3ddb7"
server_key_version: "encryptionkey@0"
I20260926 20:39:54.506633 28390 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260926 20:39:54.510623 28404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:54.512188 28390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:39:54.512440 28390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "080c50841796462985400e3a294a2efc"
format_stamp: "Formatted at 2026-09-26 20:39:54 on dist-test-slave-4d5t"
server_key: "99dc689e53e485ac35ffaf102180a9f3"
server_key_iv: "56f068695b9d198c92f0278cf4c3ddb7"
server_key_version: "encryptionkey@0"
I20260926 20:39:54.512765 28390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:54.519335 28390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:54.520146 28390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:54.520455 28390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:54.540277 28390 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:42797
I20260926 20:39:54.540342 28455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:42797 every 8 connection(s)
I20260926 20:39:54.541781 28390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:54.545629 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 28390
I20260926 20:39:54.545852 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:54.546551 23731 external_mini_cluster.cc:1428] Setting key b3f642b479ceaf861fd5853a0baa83d9
I20260926 20:39:54.547010 28456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:54.555688 28459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:54.549017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48760 (local address 127.23.44.254:42797)
0926 20:39:54.550143 (+ 1126us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:54.550155 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:54.550201 (+ 46us) server_negotiation.cc:407] Connection header received
0926 20:39:54.551781 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:54.551803 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:54.552246 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:54.552687 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:54.553953 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:54.553961 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:54.553973 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:54.553989 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:54.554050 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:54.554067 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:54.554284 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:54.554403 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 20:39:54.554738 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":427,"threads_started":1}
I20260926 20:39:54.560478 28456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc: Bootstrap starting.
I20260926 20:39:54.563621 28456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:54.564770 28456 log.cc:824] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:54.568373 28456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc: No bootstrap required, opened a new log
I20260926 20:39:54.575978 28456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "080c50841796462985400e3a294a2efc" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 42797 } }
I20260926 20:39:54.576462 28456 raft_consensus.cc:374] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:54.576608 28456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 080c50841796462985400e3a294a2efc, State: Initialized, Role: FOLLOWER
I20260926 20:39:54.577180 28456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "080c50841796462985400e3a294a2efc" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 42797 } }
I20260926 20:39:54.577376 28456 raft_consensus.cc:388] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:54.577524 28456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:54.577704 28456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:54.579856 28456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "080c50841796462985400e3a294a2efc" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 42797 } }
I20260926 20:39:54.580370 28456 leader_election.cc:302] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 080c50841796462985400e3a294a2efc; no voters:
I20260926 20:39:54.580999 28456 leader_election.cc:288] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:54.581198 28461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:54.581997 28461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [term 1 LEADER]: Becoming Leader. State: Replica: 080c50841796462985400e3a294a2efc, State: Running, Role: LEADER
I20260926 20:39:54.582568 28461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "080c50841796462985400e3a294a2efc" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 42797 } }
I20260926 20:39:54.583235 28456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:39:54.586097 28463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 080c50841796462985400e3a294a2efc. Latest consensus state: current_term: 1 leader_uuid: "080c50841796462985400e3a294a2efc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "080c50841796462985400e3a294a2efc" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 42797 } } }
I20260926 20:39:54.586018 28462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "080c50841796462985400e3a294a2efc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "080c50841796462985400e3a294a2efc" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 42797 } } }
I20260926 20:39:54.586536 28463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:54.586565 28462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:54.592270 28469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:54.598214 28469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:54.607522 28469 catalog_manager.cc:1347] Generated new cluster ID: 56c789dd932c4b13a3046db92e23338b
I20260926 20:39:54.607683 28469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:54.633831 28469 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:54.634361 28469 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:54.641206 28469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc: Generated new TSK 0
I20260926 20:39:54.642341 28469 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:39:54.659469 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:42797
--builtin_ntp_servers=127.23.44.212:39395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 20:39:55.008349 28480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:55.008729 28480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:55.008903 28480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:55.018378 28480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:55.018566 28480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:55.018777 28480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:39:55.029063 28480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:42797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:55.031167 28480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:55.033551 28480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:55.051399 28486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:55.052327 28488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:55.052817 28485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:55.053187 28480 server_base.cc:1034] running on GCE node
I20260926 20:39:55.054163 28480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:55.055763 28480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:55.057016 28480 hybrid_clock.cc:648] HybridClock initialized: now 1790455195056932 us; error 65 us; skew 500 ppm
I20260926 20:39:55.057394 28480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:39:55.060106 28480 webserver.cc:460] Webserver started at http://127.23.44.193:46401/ using document root <none> and password file <none>
I20260926 20:39:55.060938 28480 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:55.061084 28480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:55.061447 28480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:55.065001 28480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "925dca27476b4afb853b6d1806fdbcbc"
format_stamp: "Formatted at 2026-09-26 20:39:55 on dist-test-slave-4d5t"
server_key: "78e4fb92a8f1feea35a287b02ac8db66"
server_key_iv: "063e7d76f146d1209992e0ff173c579a"
server_key_version: "encryptionkey@0"
I20260926 20:39:55.065860 28480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "925dca27476b4afb853b6d1806fdbcbc"
format_stamp: "Formatted at 2026-09-26 20:39:55 on dist-test-slave-4d5t"
server_key: "78e4fb92a8f1feea35a287b02ac8db66"
server_key_iv: "063e7d76f146d1209992e0ff173c579a"
server_key_version: "encryptionkey@0"
I20260926 20:39:55.071885 28480 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260926 20:39:55.075810 28494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:55.077525 28480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:39:55.077749 28480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "925dca27476b4afb853b6d1806fdbcbc"
format_stamp: "Formatted at 2026-09-26 20:39:55 on dist-test-slave-4d5t"
server_key: "78e4fb92a8f1feea35a287b02ac8db66"
server_key_iv: "063e7d76f146d1209992e0ff173c579a"
server_key_version: "encryptionkey@0"
I20260926 20:39:55.077993 28480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:55.084110 28480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:55.085183 28480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:55.085614 28480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:55.087164 28480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:55.089434 28480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:55.089591 28480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:55.089748 28480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:55.089849 28480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:55.127230 28480 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:44437
I20260926 20:39:55.127300 28606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:44437 every 8 connection(s)
I20260926 20:39:55.129268 28480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:39:55.131182 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 28480
I20260926 20:39:55.131448 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:39:55.132170 23731 external_mini_cluster.cc:1428] Setting key 52ced1b882dbd4c01f88ad9a00e2f14c
I20260926 20:39:55.135499 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:42797
--builtin_ntp_servers=127.23.44.212:39395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 20:39:55.139938 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:55.132372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:33709 (local address 127.23.44.254:42797)
0926 20:39:55.132869 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:55.132875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:55.134313 (+ 1438us) server_negotiation.cc:407] Connection header received
0926 20:39:55.136998 (+ 2685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:55.137003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:55.137067 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:55.137178 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:55.138807 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:55.138811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:55.138815 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:55.138833 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:55.138885 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:55.138898 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:55.139015 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:55.139461 (+ 446us) server_negotiation.cc:299] Negotiation successful
0926 20:39:55.139550 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":259,"threads_started":1}
I20260926 20:39:55.141201 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:55.133047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:42797 (local address 127.23.44.193:33709)
0926 20:39:55.134183 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:55.134229 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:55.136660 (+ 2431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:55.137472 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:55.137486 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:55.137627 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:55.138404 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:39:55.138409 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:55.138465 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:39:55.138597 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:55.139007 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:55.139019 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:55.139031 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 20:39:55.139468 (+ 437us) client_negotiation.cc:240] Negotiation successful
0926 20:39:55.139770 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":322,"threads_started":1}
I20260926 20:39:55.143105 28607 heartbeater.cc:347] Connected to a master server at 127.23.44.254:42797
I20260926 20:39:55.143519 28607 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:55.144191 28607 heartbeater.cc:511] Master 127.23.44.254:42797 requested a full tablet report, sending...
I20260926 20:39:55.146525 28421 ts_manager.cc:194] Registered new tserver with Master: 925dca27476b4afb853b6d1806fdbcbc (127.23.44.193:44437)
W20260926 20:39:55.488214 28611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:55.488734 28611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:55.488917 28611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:55.497237 28611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:55.497422 28611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:55.497629 28611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:39:55.506834 28611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:42797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:55.508208 28611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:55.510010 28611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:55.523280 28617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:55.523319 28619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:55.523367 28616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:55.526132 28611 server_base.cc:1034] running on GCE node
I20260926 20:39:55.526850 28611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:55.527938 28611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:55.529186 28611 hybrid_clock.cc:648] HybridClock initialized: now 1790455195529145 us; error 59 us; skew 500 ppm
I20260926 20:39:55.529572 28611 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:39:55.531968 28611 webserver.cc:460] Webserver started at http://127.23.44.194:45489/ using document root <none> and password file <none>
I20260926 20:39:55.533211 28611 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:55.533416 28611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:55.534026 28611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:55.537288 28611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "90d6a4a815064f6b91d28d0cc938c270"
format_stamp: "Formatted at 2026-09-26 20:39:55 on dist-test-slave-4d5t"
server_key: "8abe658d6b54bcaa029049826d056530"
server_key_iv: "2af1644c8d044ecdd2f54eec3919f5a7"
server_key_version: "encryptionkey@0"
I20260926 20:39:55.538182 28611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "90d6a4a815064f6b91d28d0cc938c270"
format_stamp: "Formatted at 2026-09-26 20:39:55 on dist-test-slave-4d5t"
server_key: "8abe658d6b54bcaa029049826d056530"
server_key_iv: "2af1644c8d044ecdd2f54eec3919f5a7"
server_key_version: "encryptionkey@0"
I20260926 20:39:55.545010 28611 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 20:39:55.549604 28625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:55.551329 28611 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:39:55.551503 28611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "90d6a4a815064f6b91d28d0cc938c270"
format_stamp: "Formatted at 2026-09-26 20:39:55 on dist-test-slave-4d5t"
server_key: "8abe658d6b54bcaa029049826d056530"
server_key_iv: "2af1644c8d044ecdd2f54eec3919f5a7"
server_key_version: "encryptionkey@0"
I20260926 20:39:55.551738 28611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:55.561424 28611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:55.562477 28611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:55.562780 28611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:55.563753 28611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:55.565263 28611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:55.565356 28611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:55.565433 28611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:55.565479 28611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:55.604161 28611 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:45957
I20260926 20:39:55.604218 28737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:45957 every 8 connection(s)
I20260926 20:39:55.605733 28611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:39:55.611430 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 28611
I20260926 20:39:55.611603 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:39:55.612206 23731 external_mini_cluster.cc:1428] Setting key a0944fa7417e968028ba63a8472f4f1a
I20260926 20:39:55.615303 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:42797
--builtin_ntp_servers=127.23.44.212:39395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 20:39:55.615363 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:55.608469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:42329 (local address 127.23.44.254:42797)
0926 20:39:55.608711 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:55.608718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:55.610164 (+ 1446us) server_negotiation.cc:407] Connection header received
0926 20:39:55.612146 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:55.612151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:55.612245 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:55.612374 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:55.614444 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:55.614452 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:55.614457 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:55.614479 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:55.614528 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:55.614552 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:55.614687 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:55.614937 (+ 250us) server_negotiation.cc:299] Negotiation successful
0926 20:39:55.615043 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260926 20:39:55.616392 28740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:55.609007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:42797 (local address 127.23.44.194:42329)
0926 20:39:55.610007 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:55.610046 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:55.611817 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:55.612915 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:55.612929 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:55.613078 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:55.614049 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:39:55.614052 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:55.614104 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:39:55.614217 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:55.614679 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:55.614689 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:55.614707 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 20:39:55.615042 (+ 335us) client_negotiation.cc:240] Negotiation successful
0926 20:39:55.615340 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":322,"threads_started":1}
I20260926 20:39:55.618060 28738 heartbeater.cc:347] Connected to a master server at 127.23.44.254:42797
I20260926 20:39:55.618432 28738 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:55.619122 28738 heartbeater.cc:511] Master 127.23.44.254:42797 requested a full tablet report, sending...
I20260926 20:39:55.621062 28421 ts_manager.cc:194] Registered new tserver with Master: 90d6a4a815064f6b91d28d0cc938c270 (127.23.44.194:45957)
W20260926 20:39:55.967789 28741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:55.968118 28741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:55.968211 28741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:55.975625 28741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:55.975750 28741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:55.975862 28741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:39:55.986086 28741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:42797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:55.987577 28741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:55.989626 28741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:56.005942 28746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:56.005981 28747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:56.005966 28749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:56.008159 28741 server_base.cc:1034] running on GCE node
I20260926 20:39:56.008910 28741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:56.009966 28741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:56.011243 28741 hybrid_clock.cc:648] HybridClock initialized: now 1790455196011159 us; error 53 us; skew 500 ppm
I20260926 20:39:56.011603 28741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:39:56.014515 28741 webserver.cc:460] Webserver started at http://127.23.44.195:37357/ using document root <none> and password file <none>
I20260926 20:39:56.015280 28741 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:56.015381 28741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:56.015679 28741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:56.019006 28741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "cdb7317a362748d9bb801fed54a1ab79"
format_stamp: "Formatted at 2026-09-26 20:39:56 on dist-test-slave-4d5t"
server_key: "c69cc214710a9a5d344efe8b2488efc1"
server_key_iv: "b6e73bc1abe1df3cdf601f29f91e8431"
server_key_version: "encryptionkey@0"
I20260926 20:39:56.019879 28741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "cdb7317a362748d9bb801fed54a1ab79"
format_stamp: "Formatted at 2026-09-26 20:39:56 on dist-test-slave-4d5t"
server_key: "c69cc214710a9a5d344efe8b2488efc1"
server_key_iv: "b6e73bc1abe1df3cdf601f29f91e8431"
server_key_version: "encryptionkey@0"
I20260926 20:39:56.025979 28741 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260926 20:39:56.030297 28755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:56.031996 28741 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260926 20:39:56.032297 28741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "cdb7317a362748d9bb801fed54a1ab79"
format_stamp: "Formatted at 2026-09-26 20:39:56 on dist-test-slave-4d5t"
server_key: "c69cc214710a9a5d344efe8b2488efc1"
server_key_iv: "b6e73bc1abe1df3cdf601f29f91e8431"
server_key_version: "encryptionkey@0"
I20260926 20:39:56.032560 28741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:56.038847 28741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:56.039719 28741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:56.039999 28741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:56.041642 28741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:39:56.043229 28741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:39:56.043361 28741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:56.043506 28741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:39:56.043597 28741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:56.082208 28741 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:35261
I20260926 20:39:56.082257 28867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:35261 every 8 connection(s)
I20260926 20:39:56.083734 28741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:39:56.088953 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 28741
I20260926 20:39:56.089133 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:39:56.089722 23731 external_mini_cluster.cc:1428] Setting key ecb6e83e5b20b0771e64d4a10ea2c5eb
I20260926 20:39:56.094385 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.087423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:51473 (local address 127.23.44.254:42797)
0926 20:39:56.087632 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:56.087637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:56.089561 (+ 1924us) server_negotiation.cc:407] Connection header received
0926 20:39:56.091858 (+ 2297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:56.091862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:56.091918 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:56.092083 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:56.093752 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:56.093756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:56.093759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:56.093774 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:56.093802 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:56.093813 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:56.093904 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:56.094096 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 20:39:56.094143 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 20:39:56.095402 28870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.088000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:42797 (local address 127.23.44.195:51473)
0926 20:39:56.089373 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.089426 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:56.091412 (+ 1986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:56.092398 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:56.092412 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:56.092608 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:56.093404 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:39:56.093407 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:56.093449 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:39:56.093572 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:56.093907 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:56.093915 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:56.093922 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 20:39:56.094159 (+ 237us) client_negotiation.cc:240] Negotiation successful
0926 20:39:56.094382 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":383,"threads_started":1}
I20260926 20:39:56.096944 28868 heartbeater.cc:347] Connected to a master server at 127.23.44.254:42797
I20260926 20:39:56.097501 28868 heartbeater.cc:464] Registering TS with master...
I20260926 20:39:56.098368 28868 heartbeater.cc:511] Master 127.23.44.254:42797 requested a full tablet report, sending...
I20260926 20:39:56.100234 28421 ts_manager.cc:194] Registered new tserver with Master: cdb7317a362748d9bb801fed54a1ab79 (127.23.44.195:35261)
I20260926 20:39:56.106493 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:39:56.114043 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.111507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54948 (local address 127.23.44.254:42797)
0926 20:39:56.111692 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:56.111696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:56.112327 (+ 631us) server_negotiation.cc:407] Connection header received
0926 20:39:56.112457 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:56.112461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:56.112516 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:56.112630 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:56.113521 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:56.113525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:56.113529 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:56.113543 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:56.113578 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:56.113589 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:56.113700 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:56.113794 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 20:39:56.113840 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 20:39:56.118747 28421 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:39:56.126467 28421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:39:56.147346 28880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.141685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:45957 (local address 127.0.0.1:40210)
0926 20:39:56.142772 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.142820 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:56.142981 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:56.143990 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:56.143998 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:56.144093 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:56.145040 (+ 947us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:39:56.145047 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:56.145083 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:39:56.145184 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:56.146332 (+ 1148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:56.146347 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:56.146359 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 20:39:56.146599 (+ 240us) client_negotiation.cc:240] Negotiation successful
0926 20:39:56.146725 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":476,"threads_started":1}
I20260926 20:39:56.148658 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.141851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40210 (local address 127.23.44.194:45957)
0926 20:39:56.143515 (+ 1664us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:56.143525 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:56.143550 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:39:56.143607 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:56.143612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:56.143791 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:56.143969 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:56.145916 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:56.145931 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:56.145953 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:56.145981 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:56.146069 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:56.146099 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:56.146264 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:56.147978 (+ 1714us) server_negotiation.cc:299] Negotiation successful
0926 20:39:56.148123 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1514,"thread_start_us":1403,"threads_started":1}
I20260926 20:39:56.150177 28607 heartbeater.cc:503] Master 127.23.44.254:42797 was elected leader, sending a full tablet report...
I20260926 20:39:56.160084 28880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.141840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44437 (local address 127.0.0.1:38008)
0926 20:39:56.147667 (+ 5827us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.147692 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:56.147776 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:56.158124 (+ 10348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:56.158129 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:56.158168 (+ 39us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:56.159009 (+ 841us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:39:56.159024 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:56.159047 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:39:56.159101 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:56.159604 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:56.159609 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:56.159614 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 20:39:56.159713 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 20:39:56.159769 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5680,"thread_start_us":14342,"threads_started":1}
I20260926 20:39:56.160292 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.142551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38008 (local address 127.23.44.193:44437)
0926 20:39:56.154859 (+ 12308us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:56.154871 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:56.154905 (+ 34us) server_negotiation.cc:407] Connection header received
0926 20:39:56.154977 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:56.154987 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:56.156794 (+ 1807us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:56.156957 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:56.159346 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:56.159353 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:56.159364 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:56.159379 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:56.159463 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:56.159487 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:56.159678 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:56.159749 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 20:39:56.159864 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12093,"thread_start_us":11935,"threads_started":1}
I20260926 20:39:56.161209 28885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.145425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38992 (local address 127.23.44.195:35261)
0926 20:39:56.157419 (+ 11994us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:56.157425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:56.157457 (+ 32us) server_negotiation.cc:407] Connection header received
0926 20:39:56.157518 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:56.157524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:56.157720 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:56.157890 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:56.159506 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:56.159526 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:56.159540 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:56.159566 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:56.159625 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:56.159653 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:56.159831 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:56.160719 (+ 888us) server_negotiation.cc:299] Negotiation successful
0926 20:39:56.160812 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11839,"thread_start_us":11646,"threads_started":1}
I20260926 20:39:56.160595 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.144386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:35261 (local address 127.0.0.1:38992)
0926 20:39:56.156483 (+ 12097us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.156510 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:56.156659 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:56.157969 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:56.157975 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:56.158002 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:56.159256 (+ 1254us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:39:56.159260 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:56.159277 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:39:56.159328 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:56.159889 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:56.159897 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:56.159903 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 20:39:56.160052 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 20:39:56.160108 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11936,"thread_start_us":12363,"threads_started":1}
I20260926 20:39:56.163890 28542 tablet_service.cc:1467] Processing CreateTablet for tablet 388ad84bb07549139d1cc2d951986c77 (DEFAULT_TABLE table=test-table [id=9d7afc2208204355857a976e9482ed9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:56.161470 28673 tablet_service.cc:1467] Processing CreateTablet for tablet 388ad84bb07549139d1cc2d951986c77 (DEFAULT_TABLE table=test-table [id=9d7afc2208204355857a976e9482ed9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:56.164721 28803 tablet_service.cc:1467] Processing CreateTablet for tablet 388ad84bb07549139d1cc2d951986c77 (DEFAULT_TABLE table=test-table [id=9d7afc2208204355857a976e9482ed9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:39:56.165867 28542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388ad84bb07549139d1cc2d951986c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:56.165930 28803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388ad84bb07549139d1cc2d951986c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:56.166241 28673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388ad84bb07549139d1cc2d951986c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:56.178943 28886 tablet_bootstrap.cc:492] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79: Bootstrap starting.
I20260926 20:39:56.182883 28887 tablet_bootstrap.cc:492] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270: Bootstrap starting.
I20260926 20:39:56.182073 28888 tablet_bootstrap.cc:492] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc: Bootstrap starting.
I20260926 20:39:56.183892 28886 tablet_bootstrap.cc:654] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:56.185492 28886 log.cc:824] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:56.186501 28888 tablet_bootstrap.cc:654] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:56.186836 28887 tablet_bootstrap.cc:654] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:56.187585 28888 log.cc:824] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:56.188099 28887 log.cc:824] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:56.189134 28886 tablet_bootstrap.cc:492] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79: No bootstrap required, opened a new log
I20260926 20:39:56.189436 28886 ts_tablet_manager.cc:1397] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260926 20:39:56.190243 28888 tablet_bootstrap.cc:492] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc: No bootstrap required, opened a new log
I20260926 20:39:56.190500 28888 ts_tablet_manager.cc:1397] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 20:39:56.191908 28887 tablet_bootstrap.cc:492] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270: No bootstrap required, opened a new log
I20260926 20:39:56.192165 28887 ts_tablet_manager.cc:1397] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260926 20:39:56.198793 28886 raft_consensus.cc:348] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90d6a4a815064f6b91d28d0cc938c270" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45957 } } peers { permanent_uuid: "925dca27476b4afb853b6d1806fdbcbc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44437 } } peers { permanent_uuid: "cdb7317a362748d9bb801fed54a1ab79" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35261 } }
I20260926 20:39:56.199256 28886 raft_consensus.cc:374] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:56.199380 28886 raft_consensus.cc:729] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdb7317a362748d9bb801fed54a1ab79, State: Initialized, Role: FOLLOWER
I20260926 20:39:56.200088 28886 consensus_queue.cc:260] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90d6a4a815064f6b91d28d0cc938c270" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45957 } } peers { permanent_uuid: "925dca27476b4afb853b6d1806fdbcbc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44437 } } peers { permanent_uuid: "cdb7317a362748d9bb801fed54a1ab79" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35261 } }
I20260926 20:39:56.201329 28868 heartbeater.cc:503] Master 127.23.44.254:42797 was elected leader, sending a full tablet report...
I20260926 20:39:56.201289 28888 raft_consensus.cc:348] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90d6a4a815064f6b91d28d0cc938c270" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45957 } } peers { permanent_uuid: "925dca27476b4afb853b6d1806fdbcbc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44437 } } peers { permanent_uuid: "cdb7317a362748d9bb801fed54a1ab79" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35261 } }
I20260926 20:39:56.201779 28888 raft_consensus.cc:374] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:56.201881 28888 raft_consensus.cc:729] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 925dca27476b4afb853b6d1806fdbcbc, State: Initialized, Role: FOLLOWER
I20260926 20:39:56.202512 28888 consensus_queue.cc:260] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90d6a4a815064f6b91d28d0cc938c270" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45957 } } peers { permanent_uuid: "925dca27476b4afb853b6d1806fdbcbc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44437 } } peers { permanent_uuid: "cdb7317a362748d9bb801fed54a1ab79" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35261 } }
I20260926 20:39:56.202786 28887 raft_consensus.cc:348] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90d6a4a815064f6b91d28d0cc938c270" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45957 } } peers { permanent_uuid: "925dca27476b4afb853b6d1806fdbcbc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44437 } } peers { permanent_uuid: "cdb7317a362748d9bb801fed54a1ab79" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35261 } }
I20260926 20:39:56.203338 28887 raft_consensus.cc:374] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:56.203506 28887 raft_consensus.cc:729] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90d6a4a815064f6b91d28d0cc938c270, State: Initialized, Role: FOLLOWER
I20260926 20:39:56.203715 28886 ts_tablet_manager.cc:1428] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260926 20:39:56.204432 28887 consensus_queue.cc:260] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90d6a4a815064f6b91d28d0cc938c270" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45957 } } peers { permanent_uuid: "925dca27476b4afb853b6d1806fdbcbc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44437 } } peers { permanent_uuid: "cdb7317a362748d9bb801fed54a1ab79" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35261 } }
I20260926 20:39:56.205451 28888 ts_tablet_manager.cc:1428] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260926 20:39:56.205608 28738 heartbeater.cc:503] Master 127.23.44.254:42797 was elected leader, sending a full tablet report...
I20260926 20:39:56.206176 28887 ts_tablet_manager.cc:1428] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260926 20:39:56.248845 28892 raft_consensus.cc:482] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:39:56.249137 28892 raft_consensus.cc:504] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90d6a4a815064f6b91d28d0cc938c270" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45957 } } peers { permanent_uuid: "925dca27476b4afb853b6d1806fdbcbc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44437 } } peers { permanent_uuid: "cdb7317a362748d9bb801fed54a1ab79" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35261 } }
I20260926 20:39:56.250818 28892 leader_election.cc:288] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90d6a4a815064f6b91d28d0cc938c270 (127.23.44.194:45957), 925dca27476b4afb853b6d1806fdbcbc (127.23.44.193:44437)
I20260926 20:39:56.254247 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.251449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:44577 (local address 127.23.44.193:44437)
0926 20:39:56.251677 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:56.251686 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:56.251714 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:39:56.251959 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:56.251966 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:56.252037 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:56.252182 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:56.253269 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:56.253276 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:56.253285 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:56.253322 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:56.253367 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:56.253387 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:56.253516 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:56.253852 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 20:39:56.253928 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:39:56.254396 28870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.251207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44437 (local address 127.23.44.195:44577)
0926 20:39:56.251533 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.251557 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:56.251742 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:56.252215 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:56.252222 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:56.252250 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:56.252999 (+ 749us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:39:56.253002 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:56.253022 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:39:56.253086 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:56.253604 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:56.253611 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:56.253619 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 20:39:56.253830 (+ 211us) client_negotiation.cc:240] Negotiation successful
0926 20:39:56.253953 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186}
I20260926 20:39:56.254762 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.251312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:60109 (local address 127.23.44.194:45957)
0926 20:39:56.251509 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:56.251519 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:56.252265 (+ 746us) server_negotiation.cc:407] Connection header received
0926 20:39:56.252477 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:56.252488 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:56.252580 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:56.252681 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:56.253878 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:56.253883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:56.253890 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:56.253909 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:56.253946 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:56.253962 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:56.254121 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:56.254361 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 20:39:56.254427 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:39:56.255128 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.251307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:45957 (local address 127.23.44.195:60109)
0926 20:39:56.252133 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.252153 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:56.252323 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:56.252759 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:56.252766 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:56.252795 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:39:56.253620 (+ 825us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:39:56.253623 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:39:56.253647 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:39:56.253713 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:39:56.254195 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:39:56.254200 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:39:56.254208 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 20:39:56.254384 (+ 176us) client_negotiation.cc:240] Negotiation successful
0926 20:39:56.254444 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":446,"threads_started":1}
I20260926 20:39:56.255687 28562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388ad84bb07549139d1cc2d951986c77" candidate_uuid: "cdb7317a362748d9bb801fed54a1ab79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "925dca27476b4afb853b6d1806fdbcbc" is_pre_election: true
I20260926 20:39:56.256175 28693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388ad84bb07549139d1cc2d951986c77" candidate_uuid: "cdb7317a362748d9bb801fed54a1ab79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90d6a4a815064f6b91d28d0cc938c270" is_pre_election: true
I20260926 20:39:56.256352 28562 raft_consensus.cc:2454] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdb7317a362748d9bb801fed54a1ab79 in term 0.
I20260926 20:39:56.256707 28693 raft_consensus.cc:2454] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdb7317a362748d9bb801fed54a1ab79 in term 0.
I20260926 20:39:56.257234 28757 leader_election.cc:302] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 925dca27476b4afb853b6d1806fdbcbc, cdb7317a362748d9bb801fed54a1ab79; no voters:
I20260926 20:39:56.257793 28892 raft_consensus.cc:2781] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:39:56.257972 28892 raft_consensus.cc:482] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:39:56.258109 28892 raft_consensus.cc:3037] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:56.260767 28892 raft_consensus.cc:504] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90d6a4a815064f6b91d28d0cc938c270" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45957 } } peers { permanent_uuid: "925dca27476b4afb853b6d1806fdbcbc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44437 } } peers { permanent_uuid: "cdb7317a362748d9bb801fed54a1ab79" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35261 } }
I20260926 20:39:56.261444 28892 leader_election.cc:288] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [CANDIDATE]: Term 1 election: Requested vote from peers 90d6a4a815064f6b91d28d0cc938c270 (127.23.44.194:45957), 925dca27476b4afb853b6d1806fdbcbc (127.23.44.193:44437)
I20260926 20:39:56.262207 28693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388ad84bb07549139d1cc2d951986c77" candidate_uuid: "cdb7317a362748d9bb801fed54a1ab79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90d6a4a815064f6b91d28d0cc938c270"
I20260926 20:39:56.262454 28693 raft_consensus.cc:3037] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:56.262315 28562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388ad84bb07549139d1cc2d951986c77" candidate_uuid: "cdb7317a362748d9bb801fed54a1ab79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "925dca27476b4afb853b6d1806fdbcbc"
I20260926 20:39:56.262691 28562 raft_consensus.cc:3037] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:56.265110 28693 raft_consensus.cc:2454] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdb7317a362748d9bb801fed54a1ab79 in term 1.
I20260926 20:39:56.265197 28562 raft_consensus.cc:2454] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdb7317a362748d9bb801fed54a1ab79 in term 1.
I20260926 20:39:56.265735 28756 leader_election.cc:302] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90d6a4a815064f6b91d28d0cc938c270, cdb7317a362748d9bb801fed54a1ab79; no voters:
I20260926 20:39:56.266127 28892 raft_consensus.cc:2781] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:56.266613 28892 raft_consensus.cc:686] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [term 1 LEADER]: Becoming Leader. State: Replica: cdb7317a362748d9bb801fed54a1ab79, State: Running, Role: LEADER
I20260926 20:39:56.267119 28892 consensus_queue.cc:237] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90d6a4a815064f6b91d28d0cc938c270" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45957 } } peers { permanent_uuid: "925dca27476b4afb853b6d1806fdbcbc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44437 } } peers { permanent_uuid: "cdb7317a362748d9bb801fed54a1ab79" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35261 } }
I20260926 20:39:56.273959 28419 catalog_manager.cc:5495] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdb7317a362748d9bb801fed54a1ab79 (127.23.44.195). New cstate: current_term: 1 leader_uuid: "cdb7317a362748d9bb801fed54a1ab79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90d6a4a815064f6b91d28d0cc938c270" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "925dca27476b4afb853b6d1806fdbcbc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdb7317a362748d9bb801fed54a1ab79" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35261 } health_report { overall_health: HEALTHY } } }
I20260926 20:39:56.312093 28885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.309891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39000 (local address 127.23.44.195:35261)
0926 20:39:56.310110 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:56.310115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:56.310134 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:39:56.310359 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:56.310364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:56.310450 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:56.310543 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:56.311421 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:56.311425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:56.311429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:56.311444 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:56.311473 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:56.311485 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:56.311593 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:56.311732 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 20:39:56.311778 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:39:56.321161 28562 raft_consensus.cc:1261] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc [term 1 FOLLOWER]: Refusing update from remote peer cdb7317a362748d9bb801fed54a1ab79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:56.321447 28693 raft_consensus.cc:1261] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270 [term 1 FOLLOWER]: Refusing update from remote peer cdb7317a362748d9bb801fed54a1ab79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:39:56.322264 28896 consensus_queue.cc:1035] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "925dca27476b4afb853b6d1806fdbcbc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:56.322630 28892 consensus_queue.cc:1035] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90d6a4a815064f6b91d28d0cc938c270" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:39:56.352487 28419 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:56.352881 28419 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:39:56.356928 28419 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 080c50841796462985400e3a294a2efc: Sending DeleteTablet for 3 replicas of tablet 388ad84bb07549139d1cc2d951986c77
I20260926 20:39:56.358677 28803 tablet_service.cc:1514] Processing DeleteTablet for tablet 388ad84bb07549139d1cc2d951986c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:56 UTC) from {username='slave'} at 127.0.0.1:38992
I20260926 20:39:56.358637 28673 tablet_service.cc:1514] Processing DeleteTablet for tablet 388ad84bb07549139d1cc2d951986c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:56 UTC) from {username='slave'} at 127.0.0.1:40210
I20260926 20:39:56.359483 28542 tablet_service.cc:1514] Processing DeleteTablet for tablet 388ad84bb07549139d1cc2d951986c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:39:56 UTC) from {username='slave'} at 127.0.0.1:38008
I20260926 20:39:56.359802 28911 tablet_replica.cc:330] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270: stopping tablet replica
I20260926 20:39:56.360369 28911 raft_consensus.cc:2229] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:56.361713 28910 tablet_replica.cc:330] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79: stopping tablet replica
I20260926 20:39:56.362401 28910 raft_consensus.cc:2229] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:39:56.362641 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 28480
I20260926 20:39:56.362825 28911 raft_consensus.cc:2258] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:56.363741 28910 raft_consensus.cc:2258] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:56.366386 28911 ts_tablet_manager.cc:1905] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:56.366936 28910 ts_tablet_manager.cc:1905] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:56.374615 28911 ts_tablet_manager.cc:1918] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:39:56.374974 28911 log.cc:1196] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/388ad84bb07549139d1cc2d951986c77
I20260926 20:39:56.375645 28911 ts_tablet_manager.cc:1939] T 388ad84bb07549139d1cc2d951986c77 P 90d6a4a815064f6b91d28d0cc938c270: Deleting consensus metadata
I20260926 20:39:56.377887 28405 catalog_manager.cc:4841] TS 90d6a4a815064f6b91d28d0cc938c270 (127.23.44.194:45957): tablet 388ad84bb07549139d1cc2d951986c77 (table test-table [id=9d7afc2208204355857a976e9482ed9c]) successfully deleted
I20260926 20:39:56.378232 28910 ts_tablet_manager.cc:1918] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:39:56.378439 28910 log.cc:1196] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/388ad84bb07549139d1cc2d951986c77
I20260926 20:39:56.379204 28910 ts_tablet_manager.cc:1939] T 388ad84bb07549139d1cc2d951986c77 P cdb7317a362748d9bb801fed54a1ab79: Deleting consensus metadata
I20260926 20:39:56.380049 28912 tablet_replica.cc:330] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc: stopping tablet replica
I20260926 20:39:56.380600 28912 raft_consensus.cc:2229] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:39:56.381006 28405 catalog_manager.cc:4841] TS cdb7317a362748d9bb801fed54a1ab79 (127.23.44.195:35261): tablet 388ad84bb07549139d1cc2d951986c77 (table test-table [id=9d7afc2208204355857a976e9482ed9c]) successfully deleted
I20260926 20:39:56.381243 28912 raft_consensus.cc:2258] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:39:56.382450 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.364657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38012 (local address 127.23.44.193:44437)
0926 20:39:56.364861 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:56.364867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:56.378442 (+ 13575us) server_negotiation.cc:407] Connection header received
0926 20:39:56.378761 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:56.378767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:56.378896 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:56.378979 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:56.381555 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:56.381561 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:56.381566 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:56.381601 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:56.381640 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:56.381657 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:56.381850 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:56.382104 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 20:39:56.382160 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:39:56.383340 28602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:56.383695 28912 ts_tablet_manager.cc:1905] T 388ad84bb07549139d1cc2d951986c77 P 925dca27476b4afb853b6d1806fdbcbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:39:56.501830 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 28480
I20260926 20:39:56.519352 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 28611
W20260926 20:39:56.519347 28406 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.44.193:44437 (error 108)
I20260926 20:39:56.521842 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.521321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44437 (local address 127.0.0.1:38016)
0926 20:39:56.521508 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.521654 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260926 20:39:56.522087 28406 catalog_manager.cc:4584] TS 925dca27476b4afb853b6d1806fdbcbc (127.23.44.193:44437): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 388ad84bb07549139d1cc2d951986c77: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44437: connect: Connection refused (error 111)
I20260926 20:39:56.522862 28881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.520585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40226 (local address 127.23.44.194:45957)
0926 20:39:56.520775 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:56.520782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:56.520804 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:39:56.520870 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:56.520875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:56.520980 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:56.521094 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:56.522124 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:56.522132 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:56.522140 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:56.522155 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:56.522185 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:56.522197 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:56.522324 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:56.522528 (+ 204us) server_negotiation.cc:299] Negotiation successful
0926 20:39:56.522587 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:39:56.523578 28733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:56.575896 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.575263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44437 (local address 127.0.0.1:38032)
0926 20:39:56.575549 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.575693 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260926 20:39:56.577178 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.576747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44437 (local address 127.0.0.1:38034)
0926 20:39:56.576919 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.577014 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 20:39:56.638317 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.637788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44437 (local address 127.0.0.1:38040)
0926 20:39:56.637993 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.638136 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260926 20:39:56.638917 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 28611
I20260926 20:39:56.641872 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.639079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44437 (local address 127.0.0.1:38046)
0926 20:39:56.641541 (+ 2462us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.641664 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2293}
I20260926 20:39:56.658871 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 28741
I20260926 20:39:56.662420 28885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.660026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39008 (local address 127.23.44.195:35261)
0926 20:39:56.660213 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:56.660218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:56.660246 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:39:56.660328 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:56.660332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:56.660388 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:56.660484 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:56.661576 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:56.661582 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:56.661586 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:56.661603 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:56.661636 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:56.661649 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:56.661813 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:56.662037 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 20:39:56.662115 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:39:56.663301 28863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:56.722740 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.722172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44437 (local address 127.0.0.1:38058)
0926 20:39:56.722436 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.722554 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260926 20:39:56.723974 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.723570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44437 (local address 127.0.0.1:38074)
0926 20:39:56.723785 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.723859 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260926 20:39:56.787547 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 28741
I20260926 20:39:56.806957 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 28390
I20260926 20:39:56.808231 28451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:56.921169 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 28390
I20260926 20:39:56.921263 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:56.920796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44437 (local address 127.0.0.1:38076)
0926 20:39:56.920992 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:56.921091 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
2026-09-26T20:39:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2864 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 20:39:56.963755 23731 mini_oidc.cc:152] Starting JWKS server
I20260926 20:39:56.966590 23731 webserver.cc:460] Webserver started at https://127.0.0.1:36075/ using document root <none> and password file <none>
I20260926 20:39:56.966953 23731 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 20:39:56.967767 23731 webserver.cc:460] Webserver started at http://127.0.0.1:45077/ using document root <none> and password file <none>
2026-09-26T20:39:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:56Z Disabled control of system clock
I20260926 20:39:56.998353 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33713
--webserver_interface=127.23.44.254
--webserver_port=0
--builtin_ntp_servers=127.23.44.212:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:33713
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36075/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/testchainca.cert with env {}
W20260926 20:39:57.344202 28924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:57.344599 28924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:57.344762 28924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:57.352367 28924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:57.352531 28924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:57.352626 28924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:57.352743 28924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:57.352833 28924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 20:39:57.352923 28924 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 20:39:57.353003 28924 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36075/jwks.json
I20260926 20:39:57.363197 28924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:33713
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36075/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:57.364586 28924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:57.365938 28924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:57.378278 28929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:57.378433 28932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:57.378520 28930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:57.379530 28924 server_base.cc:1034] running on GCE node
I20260926 20:39:57.380429 28924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:57.381923 28924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:57.383137 28924 hybrid_clock.cc:648] HybridClock initialized: now 1790455197383072 us; error 59 us; skew 500 ppm
I20260926 20:39:57.383499 28924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:39:57.385776 28924 webserver.cc:460] Webserver started at http://127.23.44.254:36181/ using document root <none> and password file <none>
I20260926 20:39:57.386466 28924 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:57.386569 28924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:57.386899 28924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:57.390134 28924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "560ad41c40e247f9b400ca38e59f15f0"
format_stamp: "Formatted at 2026-09-26 20:39:57 on dist-test-slave-4d5t"
server_key: "10c851ca6166c488007a23d97071a5ee"
server_key_iv: "087168dae424167c55e7118beedb353a"
server_key_version: "encryptionkey@0"
I20260926 20:39:57.390908 28924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "560ad41c40e247f9b400ca38e59f15f0"
format_stamp: "Formatted at 2026-09-26 20:39:57 on dist-test-slave-4d5t"
server_key: "10c851ca6166c488007a23d97071a5ee"
server_key_iv: "087168dae424167c55e7118beedb353a"
server_key_version: "encryptionkey@0"
I20260926 20:39:57.395938 28924 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 20:39:57.399598 28938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:57.401038 28924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 20:39:57.401244 28924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "560ad41c40e247f9b400ca38e59f15f0"
format_stamp: "Formatted at 2026-09-26 20:39:57 on dist-test-slave-4d5t"
server_key: "10c851ca6166c488007a23d97071a5ee"
server_key_iv: "087168dae424167c55e7118beedb353a"
server_key_version: "encryptionkey@0"
I20260926 20:39:57.401480 28924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:57.450248 28924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:57.451175 28924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:57.451507 28924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:57.472899 28924 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:33713
I20260926 20:39:57.472935 28989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:33713 every 8 connection(s)
I20260926 20:39:57.474359 28924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:57.478379 28990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:57.481089 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 28924
I20260926 20:39:57.481379 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:57.482095 23731 external_mini_cluster.cc:1428] Setting key 3ae27be04b4ceea22a5009f35a5b8fc4
I20260926 20:39:57.491166 28990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0: Bootstrap starting.
I20260926 20:39:57.494791 28990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:57.495752 28990 log.cc:824] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:57.498759 28990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0: No bootstrap required, opened a new log
I20260926 20:39:57.499866 28993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:57.484372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54934 (local address 127.23.44.254:33713)
0926 20:39:57.485416 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:57.485426 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:57.485468 (+ 42us) server_negotiation.cc:407] Connection header received
0926 20:39:57.487055 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:57.487077 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:57.487696 (+ 619us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:57.488060 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:57.488781 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:57.490061 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:57.491907 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:57.492624 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:57.492770 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:57.497309 (+ 4539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:57.497317 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:57.497332 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:57.497871 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:57.497949 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:57.497972 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:57.498116 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:57.498340 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 20:39:57.498653 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":440,"threads_started":1}
I20260926 20:39:57.504029 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:39:57.506994 28990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560ad41c40e247f9b400ca38e59f15f0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33713 } }
I20260926 20:39:57.507407 28990 raft_consensus.cc:374] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:57.507519 28990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560ad41c40e247f9b400ca38e59f15f0, State: Initialized, Role: FOLLOWER
I20260926 20:39:57.508181 28990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560ad41c40e247f9b400ca38e59f15f0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33713 } }
I20260926 20:39:57.508442 28990 raft_consensus.cc:388] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:57.508612 28990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:57.508790 28990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:57.511772 28990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560ad41c40e247f9b400ca38e59f15f0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33713 } }
I20260926 20:39:57.512298 28990 leader_election.cc:302] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 560ad41c40e247f9b400ca38e59f15f0; no voters:
I20260926 20:39:57.513033 28990 leader_election.cc:288] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:57.513374 28995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:57.514321 28995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [term 1 LEADER]: Becoming Leader. State: Replica: 560ad41c40e247f9b400ca38e59f15f0, State: Running, Role: LEADER
I20260926 20:39:57.515036 28995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560ad41c40e247f9b400ca38e59f15f0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33713 } }
I20260926 20:39:57.515286 28990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:39:57.517793 28997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 560ad41c40e247f9b400ca38e59f15f0. Latest consensus state: current_term: 1 leader_uuid: "560ad41c40e247f9b400ca38e59f15f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560ad41c40e247f9b400ca38e59f15f0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33713 } } }
I20260926 20:39:57.518191 28997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:57.518743 28996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "560ad41c40e247f9b400ca38e59f15f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "560ad41c40e247f9b400ca38e59f15f0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33713 } } }
I20260926 20:39:57.519066 28996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:57.520126 29002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:57.525007 29002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:57.539067 29002 catalog_manager.cc:1347] Generated new cluster ID: 806293898fe046b48c2bffe31dbcee99
I20260926 20:39:57.539203 29002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:57.564966 29002 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:57.566962 29002 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:57.588129 29002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 560ad41c40e247f9b400ca38e59f15f0: Generated new TSK 0
I20260926 20:39:57.589190 29002 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 20:39:57.645777 29021 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 20:39:57.645975 29021 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 20:39:57.647516 28993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:57.615081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54950 (local address 127.23.44.254:33713)
0926 20:39:57.615310 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:57.615315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:57.615968 (+ 653us) server_negotiation.cc:407] Connection header received
0926 20:39:57.616427 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:57.616432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:57.616508 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:57.616741 (+ 233us) server_negotiation.cc:226] Negotiated authn=JWT
0926 20:39:57.617345 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:57.618166 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:57.619705 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:57.620200 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:57.620348 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:57.620992 (+ 644us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 20:39:57.646499 (+ 25507us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 20:39:57.647154 (+ 655us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54950: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260926 20:39:57.647667 28993 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54950: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 20:39:57.649278 29020 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:39:57.615006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33713 (local address 127.0.0.1:54950)
0926 20:39:57.615845 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:57.615868 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:57.616108 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:57.616758 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:57.616764 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:57.616777 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0926 20:39:57.617160 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:57.617170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:57.618324 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:57.618330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:57.619541 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:57.619553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:57.620396 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:57.620406 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:57.620632 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:57.620696 (+ 64us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 20:39:57.647081 (+ 26385us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 20:39:57.647568 (+ 487us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.44.254:33713: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":444,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.23.44.254:33713: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36075/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 20:39:57.651463 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 28924
I20260926 20:39:57.652796 28985 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:57.749552 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 28924
2026-09-26T20:39:57Z chronyd exiting
I20260926 20:39:57.802384 23731 test_util.cc:175] -----------------------------------------------
I20260926 20:39:57.802546 23731 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790455151607908-23731-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (843 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 20:39:57.806562 23731 mini_oidc.cc:152] Starting JWKS server
I20260926 20:39:57.808754 23731 webserver.cc:460] Webserver started at https://127.0.0.1:42875/ using document root <none> and password file <none>
I20260926 20:39:57.809010 23731 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 20:39:57.809782 23731 webserver.cc:460] Webserver started at http://127.0.0.1:45125/ using document root <none> and password file <none>
2026-09-26T20:39:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:57Z Disabled control of system clock
I20260926 20:39:57.837884 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:34883
--webserver_interface=127.23.44.254
--webserver_port=0
--builtin_ntp_servers=127.23.44.212:42377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:34883
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42875/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/testchainca.cert with env {}
W20260926 20:39:58.172231 29028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:58.172624 29028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:58.172784 29028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:58.180224 29028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:58.180387 29028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:58.180483 29028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:58.180609 29028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:58.180698 29028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 20:39:58.180788 29028 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 20:39:58.180872 29028 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42875/jwks.json
I20260926 20:39:58.191334 29028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:42377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:34883
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:34883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42875/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:58.192791 29028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:58.194121 29028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:58.206070 29034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:58.206154 29036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:58.206158 29033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:58.207373 29028 server_base.cc:1034] running on GCE node
I20260926 20:39:58.208189 29028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:58.209813 29028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:58.211076 29028 hybrid_clock.cc:648] HybridClock initialized: now 1790455198211029 us; error 35 us; skew 500 ppm
I20260926 20:39:58.211444 29028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:39:58.213918 29028 webserver.cc:460] Webserver started at http://127.23.44.254:34759/ using document root <none> and password file <none>
I20260926 20:39:58.214610 29028 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:58.214748 29028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:58.215085 29028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:58.218104 29028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "1c700f12e74348b3a160c9e5f8fbe491"
format_stamp: "Formatted at 2026-09-26 20:39:58 on dist-test-slave-4d5t"
server_key: "eb57effd3e9ec4d5ba0c22bceebcd237"
server_key_iv: "7861632c7f571cb67870d087da68d38b"
server_key_version: "encryptionkey@0"
I20260926 20:39:58.218914 29028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "1c700f12e74348b3a160c9e5f8fbe491"
format_stamp: "Formatted at 2026-09-26 20:39:58 on dist-test-slave-4d5t"
server_key: "eb57effd3e9ec4d5ba0c22bceebcd237"
server_key_iv: "7861632c7f571cb67870d087da68d38b"
server_key_version: "encryptionkey@0"
I20260926 20:39:58.224320 29028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:39:58.228157 29042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:58.229812 29028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:39:58.230032 29028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "1c700f12e74348b3a160c9e5f8fbe491"
format_stamp: "Formatted at 2026-09-26 20:39:58 on dist-test-slave-4d5t"
server_key: "eb57effd3e9ec4d5ba0c22bceebcd237"
server_key_iv: "7861632c7f571cb67870d087da68d38b"
server_key_version: "encryptionkey@0"
I20260926 20:39:58.230279 29028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:58.249517 29028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:58.250811 29028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:58.251150 29028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:58.273103 29028 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:34883
I20260926 20:39:58.273138 29093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:34883 every 8 connection(s)
I20260926 20:39:58.274580 29028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:58.278342 29094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:58.281859 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 29028
I20260926 20:39:58.282070 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:58.282709 23731 external_mini_cluster.cc:1428] Setting key c17dc5d714b4eeff90260896c496f81d
I20260926 20:39:58.290952 29094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491: Bootstrap starting.
I20260926 20:39:58.294688 29094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:58.295902 29094 log.cc:824] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:58.299081 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:58.284790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49074 (local address 127.23.44.254:34883)
0926 20:39:58.286003 (+ 1213us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:58.286018 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:58.286072 (+ 54us) server_negotiation.cc:407] Connection header received
0926 20:39:58.288818 (+ 2746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:58.288850 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:58.289657 (+ 807us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:58.290121 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:58.290660 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:58.292643 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:58.293982 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:58.295052 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:58.295261 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:58.296337 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:58.296351 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:58.296376 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:58.297102 (+ 726us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:58.297183 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:58.297204 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:58.297356 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:58.297546 (+ 190us) server_negotiation.cc:299] Negotiation successful
0926 20:39:58.297878 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":455,"threads_started":1}
I20260926 20:39:58.299360 29094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491: No bootstrap required, opened a new log
I20260926 20:39:58.303036 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:39:58.308403 29094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c700f12e74348b3a160c9e5f8fbe491" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34883 } }
I20260926 20:39:58.308833 29094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:58.308931 29094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c700f12e74348b3a160c9e5f8fbe491, State: Initialized, Role: FOLLOWER
I20260926 20:39:58.309690 29094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c700f12e74348b3a160c9e5f8fbe491" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34883 } }
I20260926 20:39:58.309919 29094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:58.310026 29094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:58.310158 29094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:58.312786 29094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c700f12e74348b3a160c9e5f8fbe491" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34883 } }
I20260926 20:39:58.313371 29094 leader_election.cc:302] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c700f12e74348b3a160c9e5f8fbe491; no voters:
I20260926 20:39:58.314126 29094 leader_election.cc:288] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:58.314296 29099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:58.316356 29094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:39:58.316895 29099 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [term 1 LEADER]: Becoming Leader. State: Replica: 1c700f12e74348b3a160c9e5f8fbe491, State: Running, Role: LEADER
I20260926 20:39:58.317513 29099 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c700f12e74348b3a160c9e5f8fbe491" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34883 } }
I20260926 20:39:58.321022 29101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c700f12e74348b3a160c9e5f8fbe491. Latest consensus state: current_term: 1 leader_uuid: "1c700f12e74348b3a160c9e5f8fbe491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c700f12e74348b3a160c9e5f8fbe491" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34883 } } }
I20260926 20:39:58.321072 29100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c700f12e74348b3a160c9e5f8fbe491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c700f12e74348b3a160c9e5f8fbe491" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34883 } } }
I20260926 20:39:58.321494 29100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:58.321494 29101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:58.323686 29110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:58.330946 29110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:58.341727 29110 catalog_manager.cc:1347] Generated new cluster ID: c09179dede604e81ba6bbed0633e9560
I20260926 20:39:58.341882 29110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:58.355614 29110 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:58.357079 29110 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:58.374029 29110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c700f12e74348b3a160c9e5f8fbe491: Generated new TSK 0
I20260926 20:39:58.374958 29110 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 20:39:58.418069 29124 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 20:39:58.418205 29124 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 20:39:58.419466 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:58.396711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49088 (local address 127.23.44.254:34883)
0926 20:39:58.397123 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:58.397136 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:58.397661 (+ 525us) server_negotiation.cc:407] Connection header received
0926 20:39:58.398167 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:58.398173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:58.398254 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:58.398418 (+ 164us) server_negotiation.cc:226] Negotiated authn=JWT
0926 20:39:58.399350 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:58.400812 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:58.403039 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:58.403569 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:58.403687 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:58.404326 (+ 639us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 20:39:58.418791 (+ 14465us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42875/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 20:39:58.419251 (+ 460us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49088: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42875/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":158}
W20260926 20:39:58.419611 29097 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49088: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42875/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260926 20:39:58.420145 29123 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:39:58.396596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:34883 (local address 127.0.0.1:49088)
0926 20:39:58.397481 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:58.397512 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:58.397848 (+ 336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:58.398469 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:58.398475 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:58.398514 (+ 39us) client_negotiation.cc:189] Negotiated authn=JWT
0926 20:39:58.399044 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:58.399077 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:58.401009 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:58.401020 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:58.402823 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:58.402840 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:58.403776 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:58.403783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:58.403975 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:58.404023 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 20:39:58.419091 (+ 15068us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42875/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 20:39:58.419394 (+ 303us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.44.254:34883: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42875/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":579,"threads_started":1}
I20260926 20:39:58.421895 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 29028
I20260926 20:39:58.423450 29089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:39:58.519882 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 29028
2026-09-26T20:39:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (769 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 20:39:58.576210 23731 mini_oidc.cc:152] Starting JWKS server
I20260926 20:39:58.578191 23731 webserver.cc:460] Webserver started at https://127.0.0.1:46363/ using document root <none> and password file <none>
I20260926 20:39:58.578467 23731 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 20:39:58.579133 23731 webserver.cc:460] Webserver started at http://127.0.0.1:44633/ using document root <none> and password file <none>
2026-09-26T20:39:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:58Z Disabled control of system clock
I20260926 20:39:58.610811 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:35669
--webserver_interface=127.23.44.254
--webserver_port=0
--builtin_ntp_servers=127.23.44.212:37837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:35669
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46363/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 20:39:58.944080 29131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:58.944466 29131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:58.944654 29131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:58.953554 29131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:58.953732 29131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:58.953821 29131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:58.953933 29131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:58.954015 29131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 20:39:58.954113 29131 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 20:39:58.954190 29131 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46363/jwks.json
I20260926 20:39:58.964898 29131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:37837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:35669
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:35669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46363/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:58.966897 29131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:58.969015 29131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:58.981421 29136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:58.981433 29139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:58.981523 29137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:58.982594 29131 server_base.cc:1034] running on GCE node
I20260926 20:39:58.983392 29131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:58.985025 29131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:58.986254 29131 hybrid_clock.cc:648] HybridClock initialized: now 1790455198986210 us; error 58 us; skew 500 ppm
I20260926 20:39:58.986650 29131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:39:58.989619 29131 webserver.cc:460] Webserver started at http://127.23.44.254:45759/ using document root <none> and password file <none>
I20260926 20:39:58.990417 29131 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:58.990511 29131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:58.990839 29131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:58.994406 29131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "24eacbdbfaa440609236ba05c720eb4a"
format_stamp: "Formatted at 2026-09-26 20:39:58 on dist-test-slave-4d5t"
server_key: "7a104d58d3c76208ece7bd4c4154b3fd"
server_key_iv: "6216647cca8d229ed38a2e1d12aacd56"
server_key_version: "encryptionkey@0"
I20260926 20:39:58.995306 29131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "24eacbdbfaa440609236ba05c720eb4a"
format_stamp: "Formatted at 2026-09-26 20:39:58 on dist-test-slave-4d5t"
server_key: "7a104d58d3c76208ece7bd4c4154b3fd"
server_key_iv: "6216647cca8d229ed38a2e1d12aacd56"
server_key_version: "encryptionkey@0"
I20260926 20:39:59.000830 29131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:39:59.004441 29145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:59.005931 29131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:39:59.006129 29131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "24eacbdbfaa440609236ba05c720eb4a"
format_stamp: "Formatted at 2026-09-26 20:39:58 on dist-test-slave-4d5t"
server_key: "7a104d58d3c76208ece7bd4c4154b3fd"
server_key_iv: "6216647cca8d229ed38a2e1d12aacd56"
server_key_version: "encryptionkey@0"
I20260926 20:39:59.006376 29131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:59.033759 29131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:59.034664 29131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:59.034996 29131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:59.064237 29131 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:35669
I20260926 20:39:59.064281 29196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:35669 every 8 connection(s)
I20260926 20:39:59.066648 29131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:59.071825 29197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:59.073276 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 29131
I20260926 20:39:59.073539 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:59.074430 23731 external_mini_cluster.cc:1428] Setting key 503a6772f9ed4822c6cd97666b7e99d7
I20260926 20:39:59.086733 29197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a: Bootstrap starting.
I20260926 20:39:59.090900 29197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:59.092278 29197 log.cc:824] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:59.096212 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:59.076663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52548 (local address 127.23.44.254:35669)
0926 20:39:59.078160 (+ 1497us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:59.078185 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:59.081260 (+ 3075us) server_negotiation.cc:407] Connection header received
0926 20:39:59.084530 (+ 3270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:59.084592 (+ 62us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:59.085312 (+ 720us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:59.085832 (+ 520us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:59.086448 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:59.088730 (+ 2282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:59.090159 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:59.091362 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:59.091605 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:59.093283 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:59.093291 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:59.093306 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:59.093900 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:59.094028 (+ 128us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:59.094057 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:59.094195 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:59.094453 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 20:39:59.094814 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":417,"threads_started":1}
I20260926 20:39:59.096376 29197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a: No bootstrap required, opened a new log
I20260926 20:39:59.101549 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:39:59.105496 29197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24eacbdbfaa440609236ba05c720eb4a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35669 } }
I20260926 20:39:59.105908 29197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:59.106047 29197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24eacbdbfaa440609236ba05c720eb4a, State: Initialized, Role: FOLLOWER
I20260926 20:39:59.106654 29197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24eacbdbfaa440609236ba05c720eb4a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35669 } }
I20260926 20:39:59.106899 29197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:59.107043 29197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:59.107255 29197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:59.109920 29197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24eacbdbfaa440609236ba05c720eb4a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35669 } }
I20260926 20:39:59.110496 29197 leader_election.cc:302] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24eacbdbfaa440609236ba05c720eb4a; no voters:
I20260926 20:39:59.111311 29197 leader_election.cc:288] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:59.111537 29202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:59.112573 29202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [term 1 LEADER]: Becoming Leader. State: Replica: 24eacbdbfaa440609236ba05c720eb4a, State: Running, Role: LEADER
I20260926 20:39:59.113304 29202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24eacbdbfaa440609236ba05c720eb4a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35669 } }
I20260926 20:39:59.113812 29197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:39:59.116092 29204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24eacbdbfaa440609236ba05c720eb4a. Latest consensus state: current_term: 1 leader_uuid: "24eacbdbfaa440609236ba05c720eb4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24eacbdbfaa440609236ba05c720eb4a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35669 } } }
I20260926 20:39:59.116545 29204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:59.118651 29210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:59.118610 29203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24eacbdbfaa440609236ba05c720eb4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24eacbdbfaa440609236ba05c720eb4a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35669 } } }
I20260926 20:39:59.119035 29203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:59.125968 29210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:59.136379 29210 catalog_manager.cc:1347] Generated new cluster ID: 381b4287ef61430b87f64acf3bdb76d0
I20260926 20:39:59.136541 29210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:59.144958 29210 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:59.146279 29210 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:59.159106 29210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24eacbdbfaa440609236ba05c720eb4a: Generated new TSK 0
I20260926 20:39:59.159981 29210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:39:59.186904 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:59.178426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52560 (local address 127.23.44.254:35669)
0926 20:39:59.178590 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:59.178595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:59.179181 (+ 586us) server_negotiation.cc:407] Connection header received
0926 20:39:59.179356 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:59.179360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:59.179429 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:59.179570 (+ 141us) server_negotiation.cc:226] Negotiated authn=JWT
0926 20:39:59.180177 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:59.181045 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:59.183039 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:59.183524 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:59.183634 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:59.184255 (+ 621us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 20:39:59.186241 (+ 1986us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46363/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 20:39:59.186583 (+ 342us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52560: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46363/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":40}
W20260926 20:39:59.187036 29200 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52560: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46363/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 20:39:59.187167 29227 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:39:59.178255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:35669 (local address 127.0.0.1:52560)
0926 20:39:59.179024 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:59.179043 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:59.179199 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:59.179616 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:59.179623 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:59.179636 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0926 20:39:59.179988 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:59.180002 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:59.181205 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:59.181212 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:59.182887 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:59.182902 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:59.183730 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:59.183737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:59.183951 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:59.183998 (+ 47us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 20:39:59.186460 (+ 2462us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46363/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 20:39:59.186722 (+ 262us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.44.254:35669: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46363/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":471,"threads_started":1}
I20260926 20:39:59.188025 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 29131
I20260926 20:39:59.189307 29192 generic_service.cc:196] Checking for leaks (request via RPC)
E20260926 20:39:59.199060 29228 webserver.cc:533] Webserver: sslize failed:
E20260926 20:39:59.199206 29228 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 20:39:59.280674 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 29131
2026-09-26T20:39:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (759 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T20:39:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:39:59Z Disabled control of system clock
I20260926 20:39:59.365767 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:39801
--webserver_interface=127.23.44.254
--webserver_port=0
--builtin_ntp_servers=127.23.44.212:40465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:39801
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/testchainca.cert with env {}
W20260926 20:39:59.701370 29233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:39:59.701689 29233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:39:59.701844 29233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:39:59.709168 29233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:39:59.709281 29233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:39:59.709354 29233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:39:59.709458 29233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:39:59.709525 29233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 20:39:59.709614 29233 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 20:39:59.709671 29233 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 20:39:59.719821 29233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:40465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:39801
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:39801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:39:59.721247 29233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:39:59.722600 29233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:39:59.734860 29241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:59.734926 29239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:39:59.734953 29238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:39:59.735831 29233 server_base.cc:1034] running on GCE node
I20260926 20:39:59.736845 29233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:39:59.738441 29233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:39:59.739689 29233 hybrid_clock.cc:648] HybridClock initialized: now 1790455199739613 us; error 58 us; skew 500 ppm
I20260926 20:39:59.740180 29233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:39:59.743733 29233 webserver.cc:460] Webserver started at http://127.23.44.254:35633/ using document root <none> and password file <none>
I20260926 20:39:59.744951 29233 fs_manager.cc:359] Metadata directory not provided
I20260926 20:39:59.745185 29233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:39:59.745581 29233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:39:59.748775 29233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "7fb794c6a05648a9af125d520fbc6c11"
format_stamp: "Formatted at 2026-09-26 20:39:59 on dist-test-slave-4d5t"
server_key: "2e06e00ffdbe86f002ab8c083ed21f06"
server_key_iv: "2b58c3359e3435f8bce4ec2d16fa38c7"
server_key_version: "encryptionkey@0"
I20260926 20:39:59.749665 29233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "7fb794c6a05648a9af125d520fbc6c11"
format_stamp: "Formatted at 2026-09-26 20:39:59 on dist-test-slave-4d5t"
server_key: "2e06e00ffdbe86f002ab8c083ed21f06"
server_key_iv: "2b58c3359e3435f8bce4ec2d16fa38c7"
server_key_version: "encryptionkey@0"
I20260926 20:39:59.755326 29233 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260926 20:39:59.759426 29247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:39:59.761140 29233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 20:39:59.761344 29233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "7fb794c6a05648a9af125d520fbc6c11"
format_stamp: "Formatted at 2026-09-26 20:39:59 on dist-test-slave-4d5t"
server_key: "2e06e00ffdbe86f002ab8c083ed21f06"
server_key_iv: "2b58c3359e3435f8bce4ec2d16fa38c7"
server_key_version: "encryptionkey@0"
I20260926 20:39:59.761588 29233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:39:59.773975 29233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:39:59.774847 29233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:39:59.775151 29233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:39:59.796777 29233 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:39801
I20260926 20:39:59.796799 29298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:39801 every 8 connection(s)
I20260926 20:39:59.798265 29233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:39:59.802098 29299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:39:59.808600 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 29233
I20260926 20:39:59.808799 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:39:59.809598 23731 external_mini_cluster.cc:1428] Setting key 042cca25d794acda2881a62214f8352c
I20260926 20:39:59.813531 29299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11: Bootstrap starting.
I20260926 20:39:59.816758 29299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11: Neither blocks nor log segments found. Creating new log.
I20260926 20:39:59.817806 29299 log.cc:824] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11: Log is configured to *not* fsync() on all Append() calls
I20260926 20:39:59.821424 29299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11: No bootstrap required, opened a new log
I20260926 20:39:59.824134 29302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:59.811634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50974 (local address 127.23.44.254:39801)
0926 20:39:59.812722 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:59.812734 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:59.812777 (+ 43us) server_negotiation.cc:407] Connection header received
0926 20:39:59.814466 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:59.814488 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:59.814954 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:59.815354 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:39:59.815904 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:59.817302 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:59.818918 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:59.820277 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:59.820479 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:59.821710 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:39:59.821719 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:39:59.821750 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:39:59.822310 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:39:59.822390 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:39:59.822411 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:39:59.822555 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 20:39:59.822778 (+ 223us) server_negotiation.cc:299] Negotiation successful
0926 20:39:59.823099 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":359,"threads_started":1}
I20260926 20:39:59.828101 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:39:59.829891 29299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb794c6a05648a9af125d520fbc6c11" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39801 } }
I20260926 20:39:59.830346 29299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:39:59.830462 29299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fb794c6a05648a9af125d520fbc6c11, State: Initialized, Role: FOLLOWER
I20260926 20:39:59.831194 29299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb794c6a05648a9af125d520fbc6c11" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39801 } }
I20260926 20:39:59.831491 29299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:39:59.831651 29299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:39:59.831837 29299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:39:59.834076 29299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb794c6a05648a9af125d520fbc6c11" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39801 } }
I20260926 20:39:59.834645 29299 leader_election.cc:302] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fb794c6a05648a9af125d520fbc6c11; no voters:
I20260926 20:39:59.835433 29299 leader_election.cc:288] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:39:59.835548 29304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:39:59.836285 29304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [term 1 LEADER]: Becoming Leader. State: Replica: 7fb794c6a05648a9af125d520fbc6c11, State: Running, Role: LEADER
I20260926 20:39:59.836911 29304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb794c6a05648a9af125d520fbc6c11" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39801 } }
I20260926 20:39:59.837841 29299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:39:59.840998 29306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fb794c6a05648a9af125d520fbc6c11. Latest consensus state: current_term: 1 leader_uuid: "7fb794c6a05648a9af125d520fbc6c11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb794c6a05648a9af125d520fbc6c11" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39801 } } }
I20260926 20:39:59.842147 29306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:59.841642 29305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fb794c6a05648a9af125d520fbc6c11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fb794c6a05648a9af125d520fbc6c11" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39801 } } }
I20260926 20:39:59.842669 29305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11 [sys.catalog]: This master's current role is: LEADER
I20260926 20:39:59.844153 29314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:39:59.850394 29314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:39:59.861948 29314 catalog_manager.cc:1347] Generated new cluster ID: 64c7e94fd5914fe7904c8eaf4d1e6728
I20260926 20:39:59.862119 29314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:39:59.878005 29314 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:39:59.879351 29314 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:39:59.893824 29314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fb794c6a05648a9af125d520fbc6c11: Generated new TSK 0
I20260926 20:39:59.894886 29314 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:40:00.353780 29302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:39:59.911110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.23.44.254:39801)
0926 20:39:59.911495 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0926 20:39:59.911517 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 20:39:59.911880 (+ 363us) server_negotiation.cc:407] Connection header received
0926 20:39:59.912191 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:39:59.912199 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:39:59.912273 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:39:59.912449 (+ 176us) server_negotiation.cc:226] Negotiated authn=JWT
0926 20:39:59.912942 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:59.913646 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:59.915056 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:39:59.915519 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:39:59.915658 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:59.916157 (+ 499us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 20:40:00.353199 (+437042us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 20:40:00.353535 (+ 336us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50978: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":76}
W20260926 20:40:00.353919 29302 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50978: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 20:40:00.354429 29328 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:39:59.910932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39801 (local address 127.0.0.1:50978)
0926 20:39:59.911767 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0926 20:39:59.911783 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:39:59.911974 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:39:59.912462 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:39:59.912466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:39:59.912478 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0926 20:39:59.912790 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:59.912799 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:59.913803 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:59.913808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:59.914904 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:39:59.914914 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:39:59.915711 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:39:59.915719 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:39:59.915860 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:39:59.915894 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 20:40:00.353581 (+437687us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 20:40:00.354118 (+ 537us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.44.254:39801: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":517,"threads_started":1}
I20260926 20:40:00.356187 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 29233
I20260926 20:40:00.357491 29294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:00.462450 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 29233
2026-09-26T20:40:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1163 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (432 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (438 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:40:01 dist-test-slave-4d5t krb5kdc[29343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:40:01 dist-test-slave-4d5t krb5kdc[29343](info): set up 2 sockets
Sep 26 20:40:01 dist-test-slave-4d5t krb5kdc[29343](info): commencing operation
krb5kdc: starting...
W20260926 20:40:02.462265 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:40:02 dist-test-slave-4d5t krb5kdc[29343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:40:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:40:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:40:02.701668 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33585
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:35715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:33585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:03.038658 29356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:03.038956 29356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:03.039112 29356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:03.050308 29356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:40:03.050479 29356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:03.050595 29356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:03.050688 29356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:40:03.050760 29356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:40:03.064038 29356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:35715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:33585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:03.065814 29356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:03.067155 29356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:03.079243 29364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:03.079627 29362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:03.079653 29361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:03.080133 29356 server_base.cc:1034] running on GCE node
I20260926 20:40:03.081018 29356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:03.082547 29356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:03.083760 29356 hybrid_clock.cc:648] HybridClock initialized: now 1790455203083721 us; error 29 us; skew 500 ppm
I20260926 20:40:03.084128 29356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:03 dist-test-slave-4d5t krb5kdc[29343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455203, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:03.087546 29356 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:40:03.089591 29356 webserver.cc:460] Webserver started at http://127.23.44.254:35843/ using document root <none> and password file <none>
I20260926 20:40:03.090385 29356 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:03.090533 29356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:03.090896 29356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:03.094247 29356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "6e5dd589ba3b4ca3b62faf0079f52873"
format_stamp: "Formatted at 2026-09-26 20:40:03 on dist-test-slave-4d5t"
server_key: "137b5c2a9f2102223d581073e2c82c38"
server_key_iv: "3a8d9b972c1aa4ba6256a4855a9c68b9"
server_key_version: "encryptionkey@0"
I20260926 20:40:03.095106 29356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "6e5dd589ba3b4ca3b62faf0079f52873"
format_stamp: "Formatted at 2026-09-26 20:40:03 on dist-test-slave-4d5t"
server_key: "137b5c2a9f2102223d581073e2c82c38"
server_key_iv: "3a8d9b972c1aa4ba6256a4855a9c68b9"
server_key_version: "encryptionkey@0"
I20260926 20:40:03.101624 29356 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260926 20:40:03.105283 29371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:03.106689 29356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:40:03.106899 29356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "6e5dd589ba3b4ca3b62faf0079f52873"
format_stamp: "Formatted at 2026-09-26 20:40:03 on dist-test-slave-4d5t"
server_key: "137b5c2a9f2102223d581073e2c82c38"
server_key_iv: "3a8d9b972c1aa4ba6256a4855a9c68b9"
server_key_version: "encryptionkey@0"
I20260926 20:40:03.107116 29356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:03.134047 29356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:03.138515 29356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:03.138819 29356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:03.158603 29356 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:33585
I20260926 20:40:03.158638 29423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:33585 every 8 connection(s)
I20260926 20:40:03.159991 29356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:40:03.163975 29424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:03.164505 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 29356
I20260926 20:40:03.164790 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:40:03.165431 23731 external_mini_cluster.cc:1428] Setting key 39517600b50b280817723a59c8e20612
I20260926 20:40:03.179085 29424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873: Bootstrap starting.
Sep 26 20:40:03 dist-test-slave-4d5t krb5kdc[29343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:40:03.183276 29424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:03.184482 29424 log.cc:824] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:03.187794 29424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873: No bootstrap required, opened a new log
I20260926 20:40:03.192066 29427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:03.167535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41076 (local address 127.23.44.254:33585)
0926 20:40:03.168405 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:03.168417 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:03.168465 (+ 48us) server_negotiation.cc:407] Connection header received
0926 20:40:03.170203 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:03.170228 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:03.171177 (+ 949us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:03.171561 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:03.173566 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:03.174948 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:03.176983 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:03.177811 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:03.177975 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:03.183157 (+ 5182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:03.183171 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:03.183195 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:03.183239 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:03.187300 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:03.188211 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:03.188218 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:03.188229 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:03.188420 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:03.188908 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:03.188912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:03.188915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:03.189498 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:03.189905 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:03.190437 (+ 532us) server_negotiation.cc:299] Negotiation successful
0926 20:40:03.190904 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":636,"thread_start_us":421,"threads_started":1}
I20260926 20:40:03.196897 29424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5dd589ba3b4ca3b62faf0079f52873" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33585 } }
I20260926 20:40:03.197365 29424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:03.197492 29424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5dd589ba3b4ca3b62faf0079f52873, State: Initialized, Role: FOLLOWER
I20260926 20:40:03.198132 29424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5dd589ba3b4ca3b62faf0079f52873" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33585 } }
I20260926 20:40:03.198410 29424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:40:03.198576 29424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:40:03.198768 29424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:03.201555 29424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5dd589ba3b4ca3b62faf0079f52873" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33585 } }
I20260926 20:40:03.202171 29424 leader_election.cc:302] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e5dd589ba3b4ca3b62faf0079f52873; no voters:
I20260926 20:40:03.202831 29424 leader_election.cc:288] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:40:03.202981 29429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:03.203758 29429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [term 1 LEADER]: Becoming Leader. State: Replica: 6e5dd589ba3b4ca3b62faf0079f52873, State: Running, Role: LEADER
I20260926 20:40:03.204700 29429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5dd589ba3b4ca3b62faf0079f52873" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33585 } }
I20260926 20:40:03.205302 29424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:40:03.208396 29430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e5dd589ba3b4ca3b62faf0079f52873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5dd589ba3b4ca3b62faf0079f52873" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33585 } } }
I20260926 20:40:03.208917 29430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:03.208796 29431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e5dd589ba3b4ca3b62faf0079f52873. Latest consensus state: current_term: 1 leader_uuid: "6e5dd589ba3b4ca3b62faf0079f52873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5dd589ba3b4ca3b62faf0079f52873" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33585 } } }
I20260926 20:40:03.209085 29431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:03.211930 29438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:40:03.216389 29438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:40:03.226455 29438 catalog_manager.cc:1347] Generated new cluster ID: 2307bb6f47af4307a62fdab5f9f234e7
I20260926 20:40:03.226576 29438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:40:03.243072 29438 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:40:03.244473 29438 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:40:03.259558 29438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e5dd589ba3b4ca3b62faf0079f52873: Generated new TSK 0
I20260926 20:40:03.260779 29438 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:40:03.329254 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33585
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:35715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:03.671046 29450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:03.671353 29450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:03.671451 29450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:03.679391 29450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:03.679529 29450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:03.679651 29450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:40:03.690068 29450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:35715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:03.691413 29450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:03.692821 29450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:03.706555 29456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:03.706727 29458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:03.706686 29455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:03.707398 29450 server_base.cc:1034] running on GCE node
I20260926 20:40:03.708057 29450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:03.709209 29450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:03.710549 29450 hybrid_clock.cc:648] HybridClock initialized: now 1790455203710464 us; error 42 us; skew 500 ppm
I20260926 20:40:03.710932 29450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:03 dist-test-slave-4d5t krb5kdc[29343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455203, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:03.714649 29450 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:40:03.716213 29450 webserver.cc:460] Webserver started at http://127.23.44.193:39325/ using document root <none> and password file <none>
I20260926 20:40:03.717053 29450 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:03.717206 29450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:03.717526 29450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:03.720500 29450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "fe47c548537e40169e2a21ced44de249"
format_stamp: "Formatted at 2026-09-26 20:40:03 on dist-test-slave-4d5t"
server_key: "7869f67bc5e349ef165f50f3de735bcb"
server_key_iv: "66d20ef87bdfe2ddb06ecc8be21423db"
server_key_version: "encryptionkey@0"
I20260926 20:40:03.721367 29450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "fe47c548537e40169e2a21ced44de249"
format_stamp: "Formatted at 2026-09-26 20:40:03 on dist-test-slave-4d5t"
server_key: "7869f67bc5e349ef165f50f3de735bcb"
server_key_iv: "66d20ef87bdfe2ddb06ecc8be21423db"
server_key_version: "encryptionkey@0"
I20260926 20:40:03.727059 29450 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260926 20:40:03.731932 29465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:03.733635 29450 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:40:03.733853 29450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "fe47c548537e40169e2a21ced44de249"
format_stamp: "Formatted at 2026-09-26 20:40:03 on dist-test-slave-4d5t"
server_key: "7869f67bc5e349ef165f50f3de735bcb"
server_key_iv: "66d20ef87bdfe2ddb06ecc8be21423db"
server_key_version: "encryptionkey@0"
I20260926 20:40:03.734119 29450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:03.761047 29450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:03.765893 29450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:03.766225 29450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:03.767297 29450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:03.768918 29450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:03.769037 29450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:03.769212 29450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:03.769296 29450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:03.808290 29450 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:43953
I20260926 20:40:03.808323 29579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:43953 every 8 connection(s)
I20260926 20:40:03.809741 29450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:40:03.813757 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 29450
I20260926 20:40:03.813982 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:40:03.814776 23731 external_mini_cluster.cc:1428] Setting key 5243dc51efc963c53c757ad9f45971e1
Sep 26 20:40:03 dist-test-slave-4d5t krb5kdc[29343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455203, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:40:03.832381 29582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:03.812717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:49315 (local address 127.23.44.254:33585)
0926 20:40:03.813232 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:03.813237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:03.814271 (+ 1034us) server_negotiation.cc:407] Connection header received
0926 20:40:03.816458 (+ 2187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:03.816464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:03.816586 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:03.816733 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:03.818358 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:03.819154 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:03.820970 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:03.821428 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:03.821610 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:03.825681 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:03.825687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:03.825691 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:03.825725 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:03.828910 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:03.829947 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:03.829959 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:03.829963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:03.830082 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:03.830552 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:03.830557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:03.830561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:03.830831 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:03.831005 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:03.831761 (+ 756us) server_negotiation.cc:299] Negotiation successful
0926 20:40:03.831998 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":290,"threads_started":1}
I20260926 20:40:03.833829 29583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:03.813097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33585 (local address 127.23.44.193:49315)
0926 20:40:03.814084 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:03.814148 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:03.816186 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:03.816983 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:03.816992 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:03.817547 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:03.818167 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:03.818186 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:03.819342 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:03.819352 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:03.820812 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:03.820826 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:03.821734 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:03.821744 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:03.822020 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:03.822809 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:03.822834 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:03.825413 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:03.829092 (+ 3679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:03.829103 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:03.829126 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:03.829767 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:03.830202 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:03.830209 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:03.830214 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:03.830427 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:03.830975 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:03.830985 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:03.831358 (+ 373us) client_negotiation.cc:769] Sending connection context
0926 20:40:03.831637 (+ 279us) client_negotiation.cc:240] Negotiation successful
0926 20:40:03.832239 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":383,"threads_started":1}
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
I20260926 20:40:03.835987 29580 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33585
I20260926 20:40:03.836297 29580 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:03.837143 29580 heartbeater.cc:511] Master 127.23.44.254:33585 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
I20260926 20:40:03.840160 29388 ts_manager.cc:194] Registered new tserver with Master: fe47c548537e40169e2a21ced44de249 (127.23.44.193:43953)
I20260926 20:40:03.842361 29388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:49315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:40:03.867592 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33585
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:35715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:04.210815 29586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:04.211179 29586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:04.211355 29586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:04.219776 29586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:04.219911 29586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:04.220108 29586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:40:04.231024 29586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:35715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:04.232385 29586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:04.233750 29586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:04.248665 29594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:04.248783 29592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:04.248694 29591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:04.249944 29586 server_base.cc:1034] running on GCE node
I20260926 20:40:04.250592 29586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:04.251587 29586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:04.252836 29586 hybrid_clock.cc:648] HybridClock initialized: now 1790455204252792 us; error 43 us; skew 500 ppm
I20260926 20:40:04.253181 29586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:04 dist-test-slave-4d5t krb5kdc[29343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455204, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:04.256484 29586 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:40:04.258068 29586 webserver.cc:460] Webserver started at http://127.23.44.194:41097/ using document root <none> and password file <none>
I20260926 20:40:04.258782 29586 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:04.258886 29586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:04.259136 29586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:04.262255 29586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "02a53ca5b22f4b09abfd9527eb19a539"
format_stamp: "Formatted at 2026-09-26 20:40:04 on dist-test-slave-4d5t"
server_key: "e02b79b148e8f662ca1342d13e068e41"
server_key_iv: "4bea8df9b696d1b503dc4d162fed8ee7"
server_key_version: "encryptionkey@0"
I20260926 20:40:04.263119 29586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "02a53ca5b22f4b09abfd9527eb19a539"
format_stamp: "Formatted at 2026-09-26 20:40:04 on dist-test-slave-4d5t"
server_key: "e02b79b148e8f662ca1342d13e068e41"
server_key_iv: "4bea8df9b696d1b503dc4d162fed8ee7"
server_key_version: "encryptionkey@0"
I20260926 20:40:04.269315 29586 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 20:40:04.273386 29601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:04.274971 29586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:40:04.275188 29586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "02a53ca5b22f4b09abfd9527eb19a539"
format_stamp: "Formatted at 2026-09-26 20:40:04 on dist-test-slave-4d5t"
server_key: "e02b79b148e8f662ca1342d13e068e41"
server_key_iv: "4bea8df9b696d1b503dc4d162fed8ee7"
server_key_version: "encryptionkey@0"
I20260926 20:40:04.275460 29586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:04.295674 29586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:04.300500 29586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:04.300907 29586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:04.302083 29586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:04.303609 29586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:04.303696 29586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:04.303829 29586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:04.303900 29586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:04.343436 29586 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:36295
I20260926 20:40:04.343472 29714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:36295 every 8 connection(s)
I20260926 20:40:04.344985 29586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:40:04.351449 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 29586
I20260926 20:40:04.351639 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:40:04.352099 23731 external_mini_cluster.cc:1428] Setting key ca01539b62c2dc48e03968fb142ca46b
Sep 26 20:40:04 dist-test-slave-4d5t krb5kdc[29343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455204, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:40:04.369355 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:04.347769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:32819 (local address 127.23.44.254:33585)
0926 20:40:04.348301 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:04.348307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:04.349623 (+ 1316us) server_negotiation.cc:407] Connection header received
0926 20:40:04.352167 (+ 2544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:04.352173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:04.352270 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:04.352399 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:04.354556 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:04.355704 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:04.357451 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:04.358216 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:04.358364 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:04.363190 (+ 4826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:04.363196 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:04.363199 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:04.363235 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:04.366064 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:04.366967 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:04.366973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:04.366976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:04.367111 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:04.367551 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:04.367557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:04.367560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:04.367839 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:04.367993 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:04.368753 (+ 760us) server_negotiation.cc:299] Negotiation successful
0926 20:40:04.368976 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":308,"threads_started":1}
I20260926 20:40:04.370471 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:04.348289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33585 (local address 127.23.44.194:32819)
0926 20:40:04.349390 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:04.349453 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:04.351812 (+ 2359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:04.352900 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:04.352914 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:04.353556 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:04.354358 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:04.354381 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:04.355888 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:04.355892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:04.357296 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:04.357310 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:04.358467 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:04.358472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:04.358698 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:04.359481 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:04.359512 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:04.362756 (+ 3244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:04.366332 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:04.366340 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:04.366356 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:04.366820 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:04.367229 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:04.367234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:04.367243 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:04.367428 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:04.367968 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:04.367977 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:04.368349 (+ 372us) client_negotiation.cc:769] Sending connection context
0926 20:40:04.368665 (+ 316us) client_negotiation.cc:240] Negotiation successful
0926 20:40:04.369234 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":469,"threads_started":1}
I20260926 20:40:04.372023 29715 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33585
I20260926 20:40:04.372344 29715 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:04.373167 29715 heartbeater.cc:511] Master 127.23.44.254:33585 requested a full tablet report, sending...
I20260926 20:40:04.374825 29388 ts_manager.cc:194] Registered new tserver with Master: 02a53ca5b22f4b09abfd9527eb19a539 (127.23.44.194:36295)
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
I20260926 20:40:04.375816 29388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:32819
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:40:04.403820 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33585
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:35715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:04.772729 29721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:04.773041 29721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:04.773217 29721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:04.780969 29721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:04.781102 29721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:04.781329 29721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:40:04.791513 29721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:35715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33585
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:04.793035 29721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:04.794449 29721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:04.809485 29727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:04.809444 29729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:04.813338 29726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:04.814620 29721 server_base.cc:1034] running on GCE node
I20260926 20:40:04.815363 29721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:04.816476 29721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:04.817762 29721 hybrid_clock.cc:648] HybridClock initialized: now 1790455204817704 us; error 64 us; skew 500 ppm
I20260926 20:40:04.818143 29721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:04 dist-test-slave-4d5t krb5kdc[29343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455204, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:04.821964 29721 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:40:04.823609 29721 webserver.cc:460] Webserver started at http://127.23.44.195:41669/ using document root <none> and password file <none>
I20260926 20:40:04.824317 29721 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:04.824451 29721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:04.824851 29721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:04.827955 29721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "366dc27233c5428c817a15626cbafb7a"
format_stamp: "Formatted at 2026-09-26 20:40:04 on dist-test-slave-4d5t"
server_key: "46fe86790fcd45a09518ce0abee3aea3"
server_key_iv: "b2224daf07a92237e3100a01dc846bfc"
server_key_version: "encryptionkey@0"
I20260926 20:40:04.829339 29721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "366dc27233c5428c817a15626cbafb7a"
format_stamp: "Formatted at 2026-09-26 20:40:04 on dist-test-slave-4d5t"
server_key: "46fe86790fcd45a09518ce0abee3aea3"
server_key_iv: "b2224daf07a92237e3100a01dc846bfc"
server_key_version: "encryptionkey@0"
I20260926 20:40:04.836172 29721 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260926 20:40:04.841318 29736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:04.843628 29721 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:40:04.843897 29721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "366dc27233c5428c817a15626cbafb7a"
format_stamp: "Formatted at 2026-09-26 20:40:04 on dist-test-slave-4d5t"
server_key: "46fe86790fcd45a09518ce0abee3aea3"
server_key_iv: "b2224daf07a92237e3100a01dc846bfc"
server_key_version: "encryptionkey@0"
I20260926 20:40:04.844195 29721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:04.846804 29580 heartbeater.cc:503] Master 127.23.44.254:33585 was elected leader, sending a full tablet report...
I20260926 20:40:04.869683 29721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:04.875321 29721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:04.875751 29721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:04.877239 29721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:04.879192 29721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:04.879352 29721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:04.879549 29721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:04.879629 29721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:04.917181 29721 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:44639
I20260926 20:40:04.917452 29849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:44639 every 8 connection(s)
I20260926 20:40:04.918624 29721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:40:04.928092 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 29721
I20260926 20:40:04.928268 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:40:04.928845 23731 external_mini_cluster.cc:1428] Setting key 6cd4ac5325e76f8abf32e42094c98489
Sep 26 20:40:04 dist-test-slave-4d5t krb5kdc[29343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455204, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:40:04.941217 29852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:04.921447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:45241 (local address 127.23.44.254:33585)
0926 20:40:04.921946 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:04.921955 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:04.922991 (+ 1036us) server_negotiation.cc:407] Connection header received
0926 20:40:04.924914 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:04.924919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:04.925002 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:04.925102 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:04.926721 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:04.927533 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:04.929953 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:04.930376 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:04.930556 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:04.934983 (+ 4427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:04.934988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:04.934992 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:04.935030 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:04.937760 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:04.938699 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:04.938709 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:04.938712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:04.938831 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:04.939345 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:04.939354 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:04.939356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:04.939657 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:04.939818 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:04.940593 (+ 775us) server_negotiation.cc:299] Negotiation successful
0926 20:40:04.940823 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":246,"threads_started":1}
I20260926 20:40:04.942230 29853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:04.922036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33585 (local address 127.23.44.195:45241)
0926 20:40:04.922812 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:04.922853 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:04.924542 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:04.925363 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:04.925372 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:04.925902 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:04.926516 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:04.926534 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:04.927713 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:04.927722 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:04.929733 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:04.929781 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:04.930689 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:04.930697 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:04.931037 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:04.931911 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:04.931935 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:04.934721 (+ 2786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:04.937951 (+ 3230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:04.937964 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:04.937984 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:04.938556 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:04.938952 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:04.938957 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:04.938963 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:04.939221 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:04.939795 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:04.939804 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:04.940183 (+ 379us) client_negotiation.cc:769] Sending connection context
0926 20:40:04.940460 (+ 277us) client_negotiation.cc:240] Negotiation successful
0926 20:40:04.941061 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":328,"threads_started":1}
I20260926 20:40:04.943984 29850 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33585
I20260926 20:40:04.944336 29850 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:04.945263 29850 heartbeater.cc:511] Master 127.23.44.254:33585 requested a full tablet report, sending...
I20260926 20:40:04.946727 29388 ts_manager.cc:194] Registered new tserver with Master: 366dc27233c5428c817a15626cbafb7a (127.23.44.195:44639)
I20260926 20:40:04.947584 29388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:45241
I20260926 20:40:04.957981 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/test-admin.keytab.
Sep 26 20:40:04 dist-test-slave-4d5t krb5kdc[29343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:04.987763 23731 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 20:40:04.993235 29858 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:04.990642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51616 (local address 127.23.44.193:43953)
0926 20:40:04.991266 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:04.991274 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:04.991301 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:40:04.991367 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:04.991374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:04.991583 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:04.991729 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:04.992681 (+ 952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51616: BlockingRecv error: recv got EOF from 127.0.0.1:51616 (error 108)
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":317,"threads_started":1}
W20260926 20:40:04.993239 23731 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/krb5cc' of type FILE
Sep 26 20:40:04 dist-test-slave-4d5t krb5kdc[29343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:04.994867 23731 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 20:40:04 dist-test-slave-4d5t krb5kdc[29343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:04.996170 23731 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 20:40:05 dist-test-slave-4d5t krb5kdc[29343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:40:05.016120 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:04.998239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51622 (local address 127.23.44.193:43953)
0926 20:40:04.998418 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:04.998423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:04.998443 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:40:04.998666 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:04.998671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:04.998746 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:04.998839 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:05.000338 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:05.001400 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:05.002758 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:05.003544 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:05.003651 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:05.008262 (+ 4611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:05.008354 (+ 92us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:05.008388 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:05.008501 (+ 113us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:05.012518 (+ 4017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:05.013453 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:05.013461 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:05.013463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:05.013611 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:05.014026 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:05.014030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:05.014032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:05.014445 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:05.014665 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:05.015224 (+ 559us) server_negotiation.cc:299] Negotiation successful
0926 20:40:05.015605 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260926 20:40:05.023291 29858 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:05.021220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51628 (local address 127.23.44.193:43953)
0926 20:40:05.021401 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:05.021406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:05.021889 (+ 483us) server_negotiation.cc:407] Connection header received
0926 20:40:05.021965 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:05.021969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:05.022037 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:05.022135 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:05.022967 (+ 832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51628: BlockingRecv error: recv got EOF from 127.0.0.1:51628 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260926 20:40:05.023676 23731 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/krb5cc' of type FILE
Sep 26 20:40:05 dist-test-slave-4d5t krb5kdc[29343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:05.025381 23731 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 20:40:05 dist-test-slave-4d5t krb5kdc[29343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:05.026789 23731 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 20:40:05 dist-test-slave-4d5t krb5kdc[29343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:40:05.043161 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:05.028369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51634 (local address 127.23.44.193:43953)
0926 20:40:05.028604 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:05.028611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:05.028835 (+ 224us) server_negotiation.cc:407] Connection header received
0926 20:40:05.029012 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:05.029016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:05.029079 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:05.029173 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:05.030186 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:05.030922 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:05.032179 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:05.032985 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:05.033144 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:05.037316 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:05.037323 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:05.037326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:05.037370 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:05.040379 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:05.041215 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:05.041219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:05.041222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:05.041372 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:05.041773 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:05.041776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:05.041778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:05.042054 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:05.042261 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:05.042635 (+ 374us) server_negotiation.cc:299] Negotiation successful
0926 20:40:05.042848 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260926 20:40:05.048591 23731 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790455151607908-23731-0/krb5kdc/krb5cc' of type FILE
W20260926 20:40:05.048606 29858 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:05.046179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51642 (local address 127.23.44.193:43953)
0926 20:40:05.046392 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:05.046401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:05.046829 (+ 428us) server_negotiation.cc:407] Connection header received
0926 20:40:05.046926 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:05.046930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:05.047033 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:05.047169 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:05.048310 (+ 1141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51642: BlockingRecv error: recv got EOF from 127.0.0.1:51642 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 26 20:40:05 dist-test-slave-4d5t krb5kdc[29343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:05.050264 23731 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 20:40:05 dist-test-slave-4d5t krb5kdc[29343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:05.051644 23731 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 20:40:05 dist-test-slave-4d5t krb5kdc[29343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:40:05.070225 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:05.053631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51650 (local address 127.23.44.193:43953)
0926 20:40:05.053843 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:05.053849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:05.054405 (+ 556us) server_negotiation.cc:407] Connection header received
0926 20:40:05.054631 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:05.054638 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:05.054728 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:05.054840 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:05.056273 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:05.057435 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:05.058856 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:05.059304 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:05.059421 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:05.063268 (+ 3847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:05.063273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:05.063277 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:05.063313 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:05.066672 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:05.067825 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:05.067829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:05.067832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:05.068050 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:05.068521 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:05.068526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:05.068529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:05.068903 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:05.069156 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:05.069558 (+ 402us) server_negotiation.cc:299] Negotiation successful
0926 20:40:05.069904 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 20:40:05.071945 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 29450
I20260926 20:40:05.085832 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:05.072737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51662 (local address 127.23.44.193:43953)
0926 20:40:05.072908 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:05.072913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:05.073221 (+ 308us) server_negotiation.cc:407] Connection header received
0926 20:40:05.073456 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:05.073460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:05.073519 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:05.073601 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:05.074506 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:05.075187 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:05.076906 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:05.077391 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:05.077502 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:05.079696 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:05.079704 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:05.079715 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:05.079749 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:05.082819 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:05.083814 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:05.083819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:05.083821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:05.083980 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:05.084436 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:05.084440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:05.084442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:05.084814 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:05.085039 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:05.085367 (+ 328us) server_negotiation.cc:299] Negotiation successful
0926 20:40:05.085554 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 20:40:05.086422 29575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:05.178498 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 29450
I20260926 20:40:05.196035 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 29586
Sep 26 20:40:05 dist-test-slave-4d5t krb5kdc[29343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:40:05.214556 29871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:05.197103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56892 (local address 127.23.44.194:36295)
0926 20:40:05.197668 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:05.197676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:05.197701 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:40:05.197760 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:05.197767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:05.197983 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:05.198127 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:05.199255 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:05.200266 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:05.202076 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:05.202691 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:05.202798 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:05.207007 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:05.207023 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:05.207041 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:05.207085 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:05.210551 (+ 3466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:05.211499 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:05.211507 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:05.211509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:05.211646 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:05.212436 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:05.212443 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:05.212445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:05.213048 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:05.213488 (+ 440us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:05.213752 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 20:40:05.214221 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":312,"threads_started":1}
I20260926 20:40:05.215960 29710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:05.302273 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 29586
I20260926 20:40:05.318542 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 29721
Sep 26 20:40:05 dist-test-slave-4d5t krb5kdc[29343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:40:05.336189 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:05.319618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55128 (local address 127.23.44.195:44639)
0926 20:40:05.320285 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:05.320293 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:05.320419 (+ 126us) server_negotiation.cc:407] Connection header received
0926 20:40:05.320491 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:05.320498 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:05.320809 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:05.320987 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:05.322058 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:05.323087 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:05.324341 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:05.325055 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:05.325196 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:05.329711 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:05.329731 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:05.329753 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:05.329814 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:05.333277 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:05.334082 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:05.334090 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:05.334093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:05.334222 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:05.334594 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:05.334601 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:05.334605 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:05.334972 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:05.335304 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:05.335597 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 20:40:05.335875 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":362,"threads_started":1}
I20260926 20:40:05.337563 29845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:05.422824 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 29721
I20260926 20:40:05.439494 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 29356
I20260926 20:40:05.440680 29419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:05.548228 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 29356
2026-09-26T20:40:05Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4220 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:40:05 dist-test-slave-4d5t krb5kdc[29878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:40:05 dist-test-slave-4d5t krb5kdc[29878](info): set up 2 sockets
Sep 26 20:40:05 dist-test-slave-4d5t krb5kdc[29878](info): commencing operation
krb5kdc: starting...
W20260926 20:40:06.658296 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:40:06 dist-test-slave-4d5t krb5kdc[29878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:40:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:40:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.254.keytab.
Entry for principal oryx/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.254.keytab.
I20260926 20:40:06.904742 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:41983
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.254.keytab
--principal=oryx/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:41983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:07.266454 29891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:07.266835 29891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:07.266975 29891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:07.275281 29891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:40:07.275403 29891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:07.275450 29891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:07.275507 29891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:40:07.275552 29891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:40:07.285738 29891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:41983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.254.keytab
--principal=oryx/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:41983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:07.287045 29891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:07.288404 29891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:07.300453 29896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:07.300523 29897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:07.300545 29899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:07.301630 29891 server_base.cc:1034] running on GCE node
I20260926 20:40:07.302457 29891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:07.303891 29891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:07.305164 29891 hybrid_clock.cc:648] HybridClock initialized: now 1790455207305089 us; error 59 us; skew 500 ppm
I20260926 20:40:07.305567 29891 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:07 dist-test-slave-4d5t krb5kdc[29878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455207, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:07.309026 29891 init.cc:377] Logged in from keytab as oryx/127.23.44.254@KRBTEST.COM (short username oryx)
I20260926 20:40:07.310819 29891 webserver.cc:460] Webserver started at http://127.23.44.254:41971/ using document root <none> and password file <none>
I20260926 20:40:07.311499 29891 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:07.311602 29891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:07.311848 29891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:07.314927 29891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "c12f5c6e8bf842719909b3f57df1bbf0"
format_stamp: "Formatted at 2026-09-26 20:40:07 on dist-test-slave-4d5t"
server_key: "bb910c152e6b8dda6719865646073fe1"
server_key_iv: "0edb43dc16f8da42ca5d0fa328f678ba"
server_key_version: "encryptionkey@0"
I20260926 20:40:07.315641 29891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "c12f5c6e8bf842719909b3f57df1bbf0"
format_stamp: "Formatted at 2026-09-26 20:40:07 on dist-test-slave-4d5t"
server_key: "bb910c152e6b8dda6719865646073fe1"
server_key_iv: "0edb43dc16f8da42ca5d0fa328f678ba"
server_key_version: "encryptionkey@0"
I20260926 20:40:07.321168 29891 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:40:07.325241 29906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:07.327104 29891 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:40:07.327378 29891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "c12f5c6e8bf842719909b3f57df1bbf0"
format_stamp: "Formatted at 2026-09-26 20:40:07 on dist-test-slave-4d5t"
server_key: "bb910c152e6b8dda6719865646073fe1"
server_key_iv: "0edb43dc16f8da42ca5d0fa328f678ba"
server_key_version: "encryptionkey@0"
I20260926 20:40:07.327621 29891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:07.355348 29891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:07.360299 29891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:07.360689 29891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:07.381091 29891 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:41983
I20260926 20:40:07.381132 29957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:41983 every 8 connection(s)
I20260926 20:40:07.382570 29891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:40:07.386464 29958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:07.389667 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 29891
I20260926 20:40:07.390051 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:40:07.390978 23731 external_mini_cluster.cc:1428] Setting key 91bb263f0441a7f04d33ac7c6c2d15cb
I20260926 20:40:07.397964 29958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0: Bootstrap starting.
I20260926 20:40:07.401851 29958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:07.403162 29958 log.cc:824] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:07.408150 29958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0: No bootstrap required, opened a new log
Sep 26 20:40:07 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
I20260926 20:40:07.419544 29958 raft_consensus.cc:348] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } }
I20260926 20:40:07.420053 29958 raft_consensus.cc:374] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:07.420193 29958 raft_consensus.cc:729] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12f5c6e8bf842719909b3f57df1bbf0, State: Initialized, Role: FOLLOWER
I20260926 20:40:07.420958 29958 consensus_queue.cc:260] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } }
I20260926 20:40:07.421262 29958 raft_consensus.cc:388] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:40:07.421298 29961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:07.393227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49600 (local address 127.23.44.254:41983)
0926 20:40:07.394198 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:07.394217 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:07.394287 (+ 70us) server_negotiation.cc:407] Connection header received
0926 20:40:07.396470 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:07.396496 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:07.397390 (+ 894us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:07.398018 (+ 628us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:07.399939 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:07.401501 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:07.403383 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:07.404497 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:07.404749 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:07.410562 (+ 5813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:07.410574 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:07.410598 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:07.410662 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:07.415580 (+ 4918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:07.416766 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:07.416778 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:07.416789 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:07.417048 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:07.417640 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:07.417646 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:07.417649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:07.418472 (+ 823us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:07.418790 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:07.419228 (+ 438us) server_negotiation.cc:299] Negotiation successful
0926 20:40:07.419715 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":371,"threads_started":1}
I20260926 20:40:07.421444 29958 raft_consensus.cc:482] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:40:07.421638 29958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:07.424467 29958 raft_consensus.cc:504] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } }
I20260926 20:40:07.425077 29958 leader_election.cc:302] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c12f5c6e8bf842719909b3f57df1bbf0; no voters:
I20260926 20:40:07.425835 29958 leader_election.cc:288] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:40:07.425953 29963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:07.426659 29963 raft_consensus.cc:686] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [term 1 LEADER]: Becoming Leader. State: Replica: c12f5c6e8bf842719909b3f57df1bbf0, State: Running, Role: LEADER
I20260926 20:40:07.427338 29963 consensus_queue.cc:237] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } }
I20260926 20:40:07.428227 29958 sys_catalog.cc:564] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:40:07.430546 29964 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } }
I20260926 20:40:07.430588 29966 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c12f5c6e8bf842719909b3f57df1bbf0. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } }
I20260926 20:40:07.430965 29964 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:07.431001 29966 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:07.433450 29970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:40:07.439445 29970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:40:07.449468 29970 catalog_manager.cc:1347] Generated new cluster ID: 7d81cd72aff04fd7b14a2d74ebb5d49b
I20260926 20:40:07.449609 29970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:40:07.476578 29970 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:40:07.477965 29970 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:40:07.491667 29970 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0: Generated new TSK 0
I20260926 20:40:07.492753 29970 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab.
Entry for principal oryx/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab.
I20260926 20:40:07.561327 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:41983
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab
--principal=oryx/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:07.918089 29985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:07.918404 29985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:07.918560 29985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:07.926759 29985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:07.927147 29985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:07.927430 29985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:40:07.941131 29985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab
--principal=oryx/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:41983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:07.943271 29985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:07.944895 29985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:07.959304 29990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:07.959263 29991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:07.959280 29993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:07.960800 29985 server_base.cc:1034] running on GCE node
I20260926 20:40:07.961483 29985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:07.962658 29985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:07.963934 29985 hybrid_clock.cc:648] HybridClock initialized: now 1790455207963865 us; error 59 us; skew 500 ppm
I20260926 20:40:07.964322 29985 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:07 dist-test-slave-4d5t krb5kdc[29878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455207, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:07.967940 29985 init.cc:377] Logged in from keytab as oryx/127.23.44.193@KRBTEST.COM (short username oryx)
I20260926 20:40:07.969735 29985 webserver.cc:460] Webserver started at http://127.23.44.193:41785/ using document root <none> and password file <none>
I20260926 20:40:07.970518 29985 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:07.970681 29985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:07.971014 29985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:07.974323 29985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "a32161d75a4447f7a69c09e3024251f1"
format_stamp: "Formatted at 2026-09-26 20:40:07 on dist-test-slave-4d5t"
server_key: "2c3ed07deb92edb758a25c0ec5d9e9b0"
server_key_iv: "298ddd82470be1216b99449bac1a3771"
server_key_version: "encryptionkey@0"
I20260926 20:40:07.975201 29985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "a32161d75a4447f7a69c09e3024251f1"
format_stamp: "Formatted at 2026-09-26 20:40:07 on dist-test-slave-4d5t"
server_key: "2c3ed07deb92edb758a25c0ec5d9e9b0"
server_key_iv: "298ddd82470be1216b99449bac1a3771"
server_key_version: "encryptionkey@0"
I20260926 20:40:07.981088 29985 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 20:40:07.985635 30000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:07.987396 29985 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:40:07.987615 29985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "a32161d75a4447f7a69c09e3024251f1"
format_stamp: "Formatted at 2026-09-26 20:40:07 on dist-test-slave-4d5t"
server_key: "2c3ed07deb92edb758a25c0ec5d9e9b0"
server_key_iv: "298ddd82470be1216b99449bac1a3771"
server_key_version: "encryptionkey@0"
I20260926 20:40:07.987854 29985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:08.013276 29985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:08.018673 29985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:08.019034 29985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:08.020119 29985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:08.021682 29985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:08.021816 29985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:08.021972 29985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:08.022054 29985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:08.061334 29985 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:35693
I20260926 20:40:08.061429 30112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:35693 every 8 connection(s)
I20260926 20:40:08.062685 29985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:40:08.066360 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 29985
I20260926 20:40:08.066679 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:40:08.067946 23731 external_mini_cluster.cc:1428] Setting key 0614fa57c1b8c79d72887624eff3c39a
Sep 26 20:40:08 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455207, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.23.44.194@KRBTEST.COM; defaulting to no policy
I20260926 20:40:08.095623 30115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:08.065510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:53061 (local address 127.23.44.254:41983)
0926 20:40:08.066012 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:08.066017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:08.067856 (+ 1839us) server_negotiation.cc:407] Connection header received
0926 20:40:08.071139 (+ 3283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:08.071145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:08.071239 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:08.071381 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:08.073193 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:08.073949 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:08.075906 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:08.076462 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:08.076663 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:08.087393 (+ 10730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:08.087401 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:08.087405 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:08.087460 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:08.091745 (+ 4285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:08.092784 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:08.092790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:08.092794 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:08.092941 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:08.093462 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:08.093468 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:08.093472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:08.093750 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:08.093867 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:08.095084 (+ 1217us) server_negotiation.cc:299] Negotiation successful
0926 20:40:08.095344 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":258,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.44.194@KRBTEST.COM" created.
I20260926 20:40:08.096963 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:08.066438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.193:53061)
0926 20:40:08.067600 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:08.067663 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:08.070764 (+ 3101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:08.071679 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:08.071688 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:08.072267 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:08.072993 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:08.073012 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:08.074137 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:08.074140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:08.075699 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:08.075719 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:08.076734 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:08.076742 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:08.077001 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:08.083465 (+ 6464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:08.083508 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:08.086929 (+ 3421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:08.092023 (+ 5094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:08.092036 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:08.092064 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:08.092629 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:08.093069 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:08.093072 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:08.093074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:08.093329 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:08.093864 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:08.093870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:08.094561 (+ 691us) client_negotiation.cc:769] Sending connection context
0926 20:40:08.094971 (+ 410us) client_negotiation.cc:240] Negotiation successful
0926 20:40:08.095543 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"thread_start_us":463,"threads_started":1}
I20260926 20:40:08.099018 30113 heartbeater.cc:347] Connected to a master server at 127.23.44.254:41983
I20260926 20:40:08.099464 30113 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:08.100451 30113 heartbeater.cc:511] Master 127.23.44.254:41983 requested a full tablet report, sending...
I20260926 20:40:08.103581 29923 ts_manager.cc:194] Registered new tserver with Master: a32161d75a4447f7a69c09e3024251f1 (127.23.44.193:35693)
I20260926 20:40:08.105312 29923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:53061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab.
Entry for principal oryx/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab.
I20260926 20:40:08.123703 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:41983
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab
--principal=oryx/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:08.475140 30119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:08.475435 30119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:08.475594 30119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:08.483284 30119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:08.483420 30119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:08.483616 30119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:40:08.494320 30119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab
--principal=oryx/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:41983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:08.495734 30119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:08.497155 30119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:08.512255 30124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:08.512212 30125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:08.512212 30127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:08.513836 30119 server_base.cc:1034] running on GCE node
I20260926 20:40:08.514779 30119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:08.516013 30119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:08.517261 30119 hybrid_clock.cc:648] HybridClock initialized: now 1790455208517205 us; error 43 us; skew 500 ppm
I20260926 20:40:08.517666 30119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:08 dist-test-slave-4d5t krb5kdc[29878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455208, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:08.521162 30119 init.cc:377] Logged in from keytab as oryx/127.23.44.194@KRBTEST.COM (short username oryx)
I20260926 20:40:08.522753 30119 webserver.cc:460] Webserver started at http://127.23.44.194:36257/ using document root <none> and password file <none>
I20260926 20:40:08.523519 30119 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:08.523672 30119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:08.523991 30119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:08.527060 30119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "a08386306b2d4fbc83729667f83d09fb"
format_stamp: "Formatted at 2026-09-26 20:40:08 on dist-test-slave-4d5t"
server_key: "aca9a01a9648c5037350ed4945f58b1f"
server_key_iv: "1ab9d38f8e24a7da57a3db24ac4e3cb5"
server_key_version: "encryptionkey@0"
I20260926 20:40:08.527881 30119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "a08386306b2d4fbc83729667f83d09fb"
format_stamp: "Formatted at 2026-09-26 20:40:08 on dist-test-slave-4d5t"
server_key: "aca9a01a9648c5037350ed4945f58b1f"
server_key_iv: "1ab9d38f8e24a7da57a3db24ac4e3cb5"
server_key_version: "encryptionkey@0"
I20260926 20:40:08.534337 30119 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260926 20:40:08.539127 30134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:08.540704 30119 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:40:08.540905 30119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "a08386306b2d4fbc83729667f83d09fb"
format_stamp: "Formatted at 2026-09-26 20:40:08 on dist-test-slave-4d5t"
server_key: "aca9a01a9648c5037350ed4945f58b1f"
server_key_iv: "1ab9d38f8e24a7da57a3db24ac4e3cb5"
server_key_version: "encryptionkey@0"
I20260926 20:40:08.541137 30119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:08.575670 30119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:08.580930 30119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:08.581256 30119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:08.582334 30119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:08.583765 30119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:08.583869 30119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:08.584033 30119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:08.584106 30119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:08.620755 30119 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:39483
I20260926 20:40:08.620816 30247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:39483 every 8 connection(s)
I20260926 20:40:08.622218 30119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:40:08.628659 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 30119
I20260926 20:40:08.628881 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:40:08.629493 23731 external_mini_cluster.cc:1428] Setting key 86838a30bc62ef29597ac7636fdfa135
Sep 26 20:40:08 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455208, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
I20260926 20:40:08.650101 30250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:08.625192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:40553 (local address 127.23.44.254:41983)
0926 20:40:08.625688 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:08.625693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:08.626626 (+ 933us) server_negotiation.cc:407] Connection header received
0926 20:40:08.628901 (+ 2275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:08.628910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:08.629113 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:08.629244 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:08.631504 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:08.632930 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:08.635955 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:08.636456 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:08.636654 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:08.642284 (+ 5630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:08.642295 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:08.642304 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:08.642363 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:08.646224 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:08.647116 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:08.647124 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:08.647130 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:08.647302 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:08.647859 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:08.647866 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:08.647872 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:08.648269 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:08.648437 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:08.649287 (+ 850us) server_negotiation.cc:299] Negotiation successful
0926 20:40:08.649572 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":278,"threads_started":1}
I20260926 20:40:08.650833 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:08.625674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.194:40553)
0926 20:40:08.626440 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:08.626489 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:08.628459 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:08.629594 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:08.629610 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:08.630463 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:08.631290 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:08.631309 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:08.633167 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:08.633176 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:08.635712 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:08.635736 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:08.636780 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:08.636790 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:08.637169 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:08.638052 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:08.638091 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:08.641910 (+ 3819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:08.646420 (+ 4510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:08.646429 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:08.646445 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:08.646966 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:08.647485 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:08.647489 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:08.647492 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:08.647713 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:08.648413 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:08.648420 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:08.648892 (+ 472us) client_negotiation.cc:769] Sending connection context
0926 20:40:08.649153 (+ 261us) client_negotiation.cc:240] Negotiation successful
0926 20:40:08.649600 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":328,"threads_started":1}
I20260926 20:40:08.652611 30248 heartbeater.cc:347] Connected to a master server at 127.23.44.254:41983
I20260926 20:40:08.652948 30248 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:08.653761 30248 heartbeater.cc:511] Master 127.23.44.254:41983 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.23.44.195@KRBTEST.COM; defaulting to no policy
I20260926 20:40:08.655498 29923 ts_manager.cc:194] Registered new tserver with Master: a08386306b2d4fbc83729667f83d09fb (127.23.44.194:39483)
I20260926 20:40:08.656466 29923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:40553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab.
Entry for principal oryx/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab.
I20260926 20:40:08.685025 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:41983
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab
--principal=oryx/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:09.032460 30254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:09.032823 30254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:09.032922 30254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:09.040614 30254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:09.040751 30254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:09.040868 30254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:40:09.055227 30254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab
--principal=oryx/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:41983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:09.056957 30254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:09.058504 30254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:09.073987 30262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:09.074095 30260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:09.074076 30259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:09.074680 30254 server_base.cc:1034] running on GCE node
I20260926 20:40:09.075327 30254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:09.076330 30254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:09.077584 30254 hybrid_clock.cc:648] HybridClock initialized: now 1790455209077523 us; error 71 us; skew 500 ppm
I20260926 20:40:09.077960 30254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:09 dist-test-slave-4d5t krb5kdc[29878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455209, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:09.081378 30254 init.cc:377] Logged in from keytab as oryx/127.23.44.195@KRBTEST.COM (short username oryx)
I20260926 20:40:09.082880 30254 webserver.cc:460] Webserver started at http://127.23.44.195:45361/ using document root <none> and password file <none>
I20260926 20:40:09.083534 30254 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:09.083644 30254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:09.083890 30254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:09.086855 30254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "e01a9a75c042415dacc31c4cfb89a56b"
format_stamp: "Formatted at 2026-09-26 20:40:09 on dist-test-slave-4d5t"
server_key: "cecf02885febca73aa55352d763394c6"
server_key_iv: "adc83d6ceeca43c0d9c2e1bb9a0cbb36"
server_key_version: "encryptionkey@0"
I20260926 20:40:09.087584 30254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "e01a9a75c042415dacc31c4cfb89a56b"
format_stamp: "Formatted at 2026-09-26 20:40:09 on dist-test-slave-4d5t"
server_key: "cecf02885febca73aa55352d763394c6"
server_key_iv: "adc83d6ceeca43c0d9c2e1bb9a0cbb36"
server_key_version: "encryptionkey@0"
I20260926 20:40:09.093295 30254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260926 20:40:09.097067 30269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:09.098634 30254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:40:09.098819 30254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "e01a9a75c042415dacc31c4cfb89a56b"
format_stamp: "Formatted at 2026-09-26 20:40:09 on dist-test-slave-4d5t"
server_key: "cecf02885febca73aa55352d763394c6"
server_key_iv: "adc83d6ceeca43c0d9c2e1bb9a0cbb36"
server_key_version: "encryptionkey@0"
I20260926 20:40:09.099045 30254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:09.108631 30113 heartbeater.cc:503] Master 127.23.44.254:41983 was elected leader, sending a full tablet report...
I20260926 20:40:09.119151 30254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:09.123369 30254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:09.123664 30254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:09.124763 30254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:09.126166 30254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:09.126269 30254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:09.126416 30254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:09.126488 30254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:09.164719 30254 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:33263
I20260926 20:40:09.164821 30381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:33263 every 8 connection(s)
I20260926 20:40:09.166105 30254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:40:09.170079 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 30254
I20260926 20:40:09.170444 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:40:09.171176 23731 external_mini_cluster.cc:1428] Setting key e4e528a275c1e059807f1f075c19beec
Sep 26 20:40:09 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455209, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
I20260926 20:40:09.186736 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.168728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:41753 (local address 127.23.44.254:41983)
0926 20:40:09.169325 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:09.169330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:09.170088 (+ 758us) server_negotiation.cc:407] Connection header received
0926 20:40:09.172146 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:09.172151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:09.172244 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:09.172367 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:09.174089 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.174923 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.176445 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.176900 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.177030 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.180786 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:09.180791 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:09.180794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:09.180825 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:09.183548 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.184326 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.184332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.184336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.184457 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.184911 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.184916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.184919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.185189 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:09.185355 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:09.186050 (+ 695us) server_negotiation.cc:299] Negotiation successful
0926 20:40:09.186302 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":348,"threads_started":1}
I20260926 20:40:09.187763 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.169212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.195:41753)
0926 20:40:09.169917 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:09.169983 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:09.171866 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:09.172670 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:09.172679 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:09.173259 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:09.173894 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.173912 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.175090 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.175095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.176305 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.176315 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.177125 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.177132 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.177357 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.178059 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:09.178085 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:09.180516 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:09.183726 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.183736 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.183761 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.184193 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.184597 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.184604 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.184608 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.184791 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.185328 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:09.185337 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:09.185700 (+ 363us) client_negotiation.cc:769] Sending connection context
0926 20:40:09.185970 (+ 270us) client_negotiation.cc:240] Negotiation successful
0926 20:40:09.186541 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":301,"threads_started":1}
I20260926 20:40:09.189221 30382 heartbeater.cc:347] Connected to a master server at 127.23.44.254:41983
I20260926 20:40:09.189546 30382 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:09.190290 30382 heartbeater.cc:511] Master 127.23.44.254:41983 requested a full tablet report, sending...
I20260926 20:40:09.191816 29923 ts_manager.cc:194] Registered new tserver with Master: e01a9a75c042415dacc31c4cfb89a56b (127.23.44.195:33263)
I20260926 20:40:09.192731 29923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:41753
I20260926 20:40:09.200932 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.44.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.44.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.44.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.253.keytab.
Entry for principal oryx/127.23.44.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.253.keytab.
I20260926 20:40:09.252267 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.253:40781
--webserver_interface=127.23.44.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.253.keytab
--principal=oryx/127.23.44.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:41983,127.23.44.253:40781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:09.598605 30388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:09.598906 30388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:09.599054 30388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:09.606644 30388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:40:09.606765 30388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:09.606846 30388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:09.606950 30388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:40:09.607031 30388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:40:09.617100 30388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:41983,127.23.44.253:40781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.253.keytab
--principal=oryx/127.23.44.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.253:40781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.44.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
W20260926 20:40:09.617528 30388 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.23.44.254:41983,127.23.44.253:40781'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 20:40:09.618624 30388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:09.619990 30388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:09.633618 30393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:09.633616 30394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:09.634181 30396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:09.635329 30388 server_base.cc:1034] running on GCE node
I20260926 20:40:09.636245 30388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:09.637784 30388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:09.639056 30388 hybrid_clock.cc:648] HybridClock initialized: now 1790455209638973 us; error 48 us; skew 500 ppm
I20260926 20:40:09.639413 30388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:09 dist-test-slave-4d5t krb5kdc[29878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455209, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:09.642992 30388 init.cc:377] Logged in from keytab as oryx/127.23.44.253@KRBTEST.COM (short username oryx)
I20260926 20:40:09.644836 30388 webserver.cc:460] Webserver started at http://127.23.44.253:34263/ using document root <none> and password file <none>
I20260926 20:40:09.645674 30388 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:09.645848 30388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:09.646328 30388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:09.650086 30388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/data/instance:
uuid: "a6a97c669fd2448c8c74d85265130e1e"
format_stamp: "Formatted at 2026-09-26 20:40:09 on dist-test-slave-4d5t"
server_key: "09c29719423dcefe0cc6aa53bb606e84"
server_key_iv: "39e770cb555669dfca65ddafc7623585"
server_key_version: "encryptionkey@0"
I20260926 20:40:09.651664 30388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/wal/instance:
uuid: "a6a97c669fd2448c8c74d85265130e1e"
format_stamp: "Formatted at 2026-09-26 20:40:09 on dist-test-slave-4d5t"
server_key: "09c29719423dcefe0cc6aa53bb606e84"
server_key_iv: "39e770cb555669dfca65ddafc7623585"
server_key_version: "encryptionkey@0"
I20260926 20:40:09.657496 30388 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 20:40:09.659364 30248 heartbeater.cc:503] Master 127.23.44.254:41983 was elected leader, sending a full tablet report...
I20260926 20:40:09.661621 30403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:09.663065 30388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:40:09.663211 30388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/wal
uuid: "a6a97c669fd2448c8c74d85265130e1e"
format_stamp: "Formatted at 2026-09-26 20:40:09 on dist-test-slave-4d5t"
server_key: "09c29719423dcefe0cc6aa53bb606e84"
server_key_iv: "39e770cb555669dfca65ddafc7623585"
server_key_version: "encryptionkey@0"
I20260926 20:40:09.663395 30388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:09.690225 30388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:09.694644 30388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:09.694875 30388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:09.716648 30388 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.253:40781
I20260926 20:40:09.716738 30454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.253:40781 every 8 connection(s)
I20260926 20:40:09.718138 30388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/data/info.pb
I20260926 20:40:09.722036 30455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:09.726006 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 30388
I20260926 20:40:09.729785 30455 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:40:09 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455209, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.253@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
Sep 26 20:40:09 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.44.253@KRBTEST.COM
I20260926 20:40:09.750939 30460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.733952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.23.44.254:41983)
0926 20:40:09.735256 (+ 1304us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:09.735261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:09.735277 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:40:09.735324 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:09.735327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:09.735381 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:09.735465 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:09.737162 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.737894 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.739288 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.739691 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.739825 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.743822 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:09.743828 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:09.743832 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:09.743871 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:09.746888 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.747814 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.747820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.747825 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.747952 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.748416 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.748424 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.748430 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.748960 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:09.749129 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:09.750181 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0926 20:40:09.750419 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1216,"thread_start_us":1062,"threads_started":1}
I20260926 20:40:09.752173 30459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.733750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:49606)
0926 20:40:09.734653 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:09.734731 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:09.734974 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:09.735576 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:09.735584 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:09.736513 (+ 929us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:09.736981 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.736999 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.738057 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.738069 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.739139 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.739151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.739914 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.739922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.740082 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.740720 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:09.740742 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:09.743491 (+ 2749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:09.747051 (+ 3560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.747063 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.747078 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.747673 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.748072 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.748077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.748082 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.748289 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.749130 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:09.749139 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:09.749706 (+ 567us) client_negotiation.cc:769] Sending connection context
0926 20:40:09.750061 (+ 355us) client_negotiation.cc:240] Negotiation successful
0926 20:40:09.750637 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":48,"thread_start_us":333,"threads_started":1}
I20260926 20:40:09.752193 30458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.728618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54680 (local address 127.23.44.253:40781)
0926 20:40:09.729575 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:09.729588 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:09.729630 (+ 42us) server_negotiation.cc:407] Connection header received
0926 20:40:09.731823 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:09.731852 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:09.732344 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:09.732776 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:09.735146 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.737138 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.738415 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.739124 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.739333 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.744174 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:09.744183 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:09.744196 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:09.744232 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:09.747599 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.748345 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.748361 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.748363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.748512 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.749108 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.749114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.749120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.749685 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:09.750015 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:09.750349 (+ 334us) server_negotiation.cc:299] Negotiation successful
0926 20:40:09.750740 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":207,"server-negotiator.queue_time_us":683,"thread_start_us":421,"threads_started":1}
I20260926 20:40:09.754376 30455 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:40:09 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455209, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.253@KRBTEST.COM for oryx/127.23.44.253@KRBTEST.COM
I20260926 20:40:09.768045 30458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.755461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54682 (local address 127.23.44.253:40781)
0926 20:40:09.755669 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:09.755676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:09.755764 (+ 88us) server_negotiation.cc:407] Connection header received
0926 20:40:09.755935 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:09.755941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:09.756006 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:09.756100 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:09.756790 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.757484 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.758620 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.759025 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.759156 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.762418 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:09.762423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:09.762426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:09.762452 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:09.765175 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.765848 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.765856 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.765860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.765999 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.766395 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.766400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.766405 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.766767 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:09.766935 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:09.767314 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 20:40:09.767694 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":163,"server-negotiator.queue_time_us":89}
I20260926 20:40:09.768044 30459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.755458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:40781 (local address 127.0.0.1:54682)
0926 20:40:09.755669 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:09.755689 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:09.755828 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:09.756114 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:09.756118 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:09.756473 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:09.756668 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.756676 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.757620 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.757625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.758475 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.758485 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.759161 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.759166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.759365 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.759945 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:09.759964 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:09.762222 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:09.765325 (+ 3103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.765333 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.765338 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.765723 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.766126 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.766131 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.766135 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.766275 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.766907 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:09.766912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:09.767058 (+ 146us) client_negotiation.cc:769] Sending connection context
0926 20:40:09.767228 (+ 170us) client_negotiation.cc:240] Negotiation successful
0926 20:40:09.767546 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 20:40:09.773758 30455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: Bootstrap starting.
I20260926 20:40:09.776842 30455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:09.777824 30455 log.cc:824] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:09.780682 30455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: No bootstrap required, opened a new log
I20260926 20:40:09.787595 30455 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } }
I20260926 20:40:09.787916 30455 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:09.788038 30455 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6a97c669fd2448c8c74d85265130e1e, State: Initialized, Role: FOLLOWER
I20260926 20:40:09.788674 30455 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } }
I20260926 20:40:09.789609 30462 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } } }
I20260926 20:40:09.789981 30462 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:40:09.790586 30455 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: configured and running, proceeding with master startup.
W20260926 20:40:09.801095 30473 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:40:09.801251 30473 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:40:09.804869 30388 master_runner.cc:428] Detected that this master a6a97c669fd2448c8c74d85265130e1e is joining an existing cluster
I20260926 20:40:09.805001 30388 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.44.253:40781
WARNING: no policy specified for oryx/127.23.44.252@KRBTEST.COM; defaulting to no policy
I20260926 20:40:09.834292 30460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.809902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49614 (local address 127.23.44.254:41983)
0926 20:40:09.810116 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:09.810123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:09.810823 (+ 700us) server_negotiation.cc:407] Connection header received
0926 20:40:09.811171 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:09.811177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:09.811281 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:09.811366 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:09.813537 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.815160 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.818015 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.818704 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.818790 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.821765 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:09.821772 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:09.821778 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:09.821820 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:09.826457 (+ 4637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.830748 (+ 4291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.830756 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.830844 (+ 88us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.831168 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.831738 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.831744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.831747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.832317 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:09.832466 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:09.833492 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0926 20:40:09.833831 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:40:09.834329 30458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.810999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54688 (local address 127.23.44.253:40781)
0926 20:40:09.811176 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:09.811182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:09.817176 (+ 5994us) server_negotiation.cc:407] Connection header received
0926 20:40:09.817389 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:09.817394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:09.817458 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:09.817524 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:09.819029 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.820245 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.822791 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.823729 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.823860 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.826688 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:09.826693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:09.826698 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:09.826736 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:09.830379 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.831505 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.831509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.831512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.831627 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.832017 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.832022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.832025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.832480 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:09.832656 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:09.833498 (+ 842us) server_negotiation.cc:299] Negotiation successful
0926 20:40:09.833820 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:40:09.835510 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.811243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:40781 (local address 127.0.0.1:54688)
0926 20:40:09.817076 (+ 5833us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:09.817097 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:09.817271 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:09.817738 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:09.817743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:09.818519 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:09.818865 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.818877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.820506 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.820513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.822347 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.822364 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.824279 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.824289 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.824533 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.825256 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:09.825295 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:09.826504 (+ 1209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:09.830522 (+ 4018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.830535 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.830539 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.831383 (+ 844us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.831752 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.831756 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.831760 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.831910 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.832725 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:09.832798 (+ 73us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:09.832980 (+ 182us) client_negotiation.cc:769] Sending connection context
0926 20:40:09.834515 (+ 1535us) client_negotiation.cc:240] Negotiation successful
0926 20:40:09.834905 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5677,"spinlock_wait_cycles":35840,"thread_start_us":5478,"threads_started":1}
I20260926 20:40:09.835678 30479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.809936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:49614)
0926 20:40:09.810711 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:09.810736 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:09.810962 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:09.811656 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:09.811664 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:09.812677 (+ 1013us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:09.813303 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.813321 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.815419 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.815429 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.817773 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.817791 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.819113 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.819119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.819316 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.820105 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:09.820128 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:09.821491 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:09.826640 (+ 5149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.826645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.826649 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.830577 (+ 3928us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.831372 (+ 795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.831376 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.831380 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.831557 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.832573 (+ 1016us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:09.832580 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:09.832980 (+ 400us) client_negotiation.cc:769] Sending connection context
0926 20:40:09.834564 (+ 1584us) client_negotiation.cc:240] Negotiation successful
0926 20:40:09.835116 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":188,"spinlock_wait_cycles":13824,"thread_start_us":434,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.44.252@KRBTEST.COM" created.
Sep 26 20:40:09 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455207, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.254@KRBTEST.COM for oryx/127.23.44.253@KRBTEST.COM
I20260926 20:40:09.861318 30458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.842235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.23.44.253:40781)
0926 20:40:09.842446 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:09.842456 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:09.845820 (+ 3364us) server_negotiation.cc:407] Connection header received
0926 20:40:09.846293 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:09.846298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:09.846394 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:09.846524 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:09.847707 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.848496 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.850073 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.850568 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.850741 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.854450 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:09.854456 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:09.854460 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:09.854496 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:09.857645 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.858469 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.858478 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.858488 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.858640 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.859049 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.859057 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.859060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.859570 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:09.859910 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:09.860691 (+ 781us) server_negotiation.cc:299] Negotiation successful
0926 20:40:09.860933 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 20:40:09.861392 30482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.842220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:40781 (local address 127.0.0.1:54692)
0926 20:40:09.845664 (+ 3444us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:09.845708 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:09.845977 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:09.846568 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:09.846576 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:09.847291 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:09.847520 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.847536 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.848735 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.848744 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.849864 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.849876 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.850767 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.850774 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.850995 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.851745 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:09.851765 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:09.854211 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:09.857834 (+ 3623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.857850 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.857859 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.858322 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.858767 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.858774 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.858778 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.858931 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.859895 (+ 964us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:09.859908 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:09.860174 (+ 266us) client_negotiation.cc:769] Sending connection context
0926 20:40:09.860543 (+ 369us) client_negotiation.cc:240] Negotiation successful
0926 20:40:09.860941 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3182,"thread_start_us":3020,"threads_started":1}
I20260926 20:40:09.863530 29923 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.44.253:40781: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e"
member_type: NON_VOTER
last_known_addr {
host: "127.23.44.253"
port: 40781
}
attrs {
promote: true
}
}
dest_uuid: "c12f5c6e8bf842719909b3f57df1bbf0"
cas_config_opid_index: -1
I20260926 20:40:09.864804 29923 consensus_queue.cc:237] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true } }
I20260926 20:40:09.868798 30486 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a6a97c669fd2448c8c74d85265130e1e (127.23.44.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true } } }
I20260926 20:40:09.870888 30430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.44.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.252.keytab.
Entry for principal oryx/127.23.44.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.252.keytab.
I20260926 20:40:09.874655 30486 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.44.253:40781
I20260926 20:40:09.874809 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.252:34937
--webserver_interface=127.23.44.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.252.keytab
--principal=oryx/127.23.44.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:41983,127.23.44.253:40781,127.23.44.252:34937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:40:09.875438 30388 master.cc:537] Master@127.23.44.253:40781 shutting down...
I20260926 20:40:09.875696 30485 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true } } }
I20260926 20:40:09.876034 30485 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:09.879659 30430 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 FOLLOWER]: Refusing update from remote peer c12f5c6e8bf842719909b3f57df1bbf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 20:40:09.881976 30486 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 20:40:09.885708 29910 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260926 20:40:09.889349 29910 consensus_peers.cc:487] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 -> Peer a6a97c669fd2448c8c74d85265130e1e (127.23.44.253:40781): Couldn't send request to peer a6a97c669fd2448c8c74d85265130e1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 20:40:09.891707 30388 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:09.892160 30388 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:09.892335 30388 tablet_replica.cc:330] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: stopping tablet replica
I20260926 20:40:09.898872 30388 master.cc:552] Master@127.23.44.253:40781 shutdown complete.
I20260926 20:40:09.898984 30388 master_runner.cc:305] Clearing existing system tablet
I20260926 20:40:09.899358 30388 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:09.899464 30388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:09.902230 30388 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260926 20:40:09.903978 30491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:09.904278 30388 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:09.904595 30388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260926 20:40:09.904695 30388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/wal
uuid: "a6a97c669fd2448c8c74d85265130e1e"
format_stamp: "Formatted at 2026-09-26 20:40:09 on dist-test-slave-4d5t"
server_key: "09c29719423dcefe0cc6aa53bb606e84"
server_key_iv: "39e770cb555669dfca65ddafc7623585"
server_key_version: "encryptionkey@0"
I20260926 20:40:09.908910 30388 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:09.913661 30388 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 20:40:09.913905 30388 log.cc:1196] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 20:40:09.914371 30388 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: Deleting consensus metadata
I20260926 20:40:09.914628 30388 master_runner.cc:315] Copying system tablet from 127.23.44.254:41983
I20260926 20:40:09.917186 30388 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: tablet copy: Beginning tablet copy session from remote peer at address 127.23.44.254:41983
I20260926 20:40:09.932595 30460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.917975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49626 (local address 127.23.44.254:41983)
0926 20:40:09.918169 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:09.918177 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:09.921106 (+ 2929us) server_negotiation.cc:407] Connection header received
0926 20:40:09.921348 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:09.921352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:09.921414 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:09.921535 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:09.922362 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.923085 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.924337 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.924860 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.924996 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.926764 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:09.926769 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:09.926772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:09.926801 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:09.929811 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.930434 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.930440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.930444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.930597 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:09.930966 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:09.930973 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:09.930976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:09.931396 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:09.931568 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:09.931933 (+ 365us) server_negotiation.cc:299] Negotiation successful
0926 20:40:09.932140 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:40:09.932763 30497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:09.917895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:49626)
0926 20:40:09.920979 (+ 3084us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:09.921007 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:09.921187 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:09.921555 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:09.921560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:09.922009 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:09.922217 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.922227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.923223 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.923229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.924189 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:09.924200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:09.925022 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:09.925028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:09.925209 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:09.925847 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:09.925865 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:09.926557 (+ 692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:09.929942 (+ 3385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.929948 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.929953 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.930307 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.930710 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:09.930716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:09.930719 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:09.930853 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:09.931520 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:09.931525 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:09.931691 (+ 166us) client_negotiation.cc:769] Sending connection context
0926 20:40:09.931835 (+ 144us) client_negotiation.cc:240] Negotiation successful
0926 20:40:09.932140 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2951,"thread_start_us":2841,"threads_started":1}
I20260926 20:40:09.934005 29943 tablet_copy_service.cc:140] P c12f5c6e8bf842719909b3f57df1bbf0: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a6a97c669fd2448c8c74d85265130e1e ({username='oryx', principal='oryx/127.23.44.253@KRBTEST.COM'} at 127.0.0.1:49626)
I20260926 20:40:09.934293 29943 tablet_copy_service.cc:161] P c12f5c6e8bf842719909b3f57df1bbf0: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a6a97c669fd2448c8c74d85265130e1e at {username='oryx', principal='oryx/127.23.44.253@KRBTEST.COM'} at 127.0.0.1:49626: session id = a6a97c669fd2448c8c74d85265130e1e-00000000000000000000000000000000
I20260926 20:40:09.937476 29943 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 20:40:09.939349 30388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 20:40:09.943917 30388 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: tablet copy: Starting download of 0 data blocks...
I20260926 20:40:09.944231 30388 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: tablet copy: Starting download of 1 WAL segments...
I20260926 20:40:09.946790 30388 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 20:40:09.950135 29943 tablet_copy_service.cc:342] P c12f5c6e8bf842719909b3f57df1bbf0: Request end of tablet copy session a6a97c669fd2448c8c74d85265130e1e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.44.253@KRBTEST.COM'} at 127.0.0.1:49626
I20260926 20:40:09.950335 29943 tablet_copy_service.cc:433] P c12f5c6e8bf842719909b3f57df1bbf0: ending tablet copy session a6a97c669fd2448c8c74d85265130e1e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a6a97c669fd2448c8c74d85265130e1e
I20260926 20:40:09.960951 30388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:09.963765 30498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:09.973691 30499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:09.974663 30501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:09.975046 30388 server_base.cc:1034] running on GCE node
I20260926 20:40:09.975639 30388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:09.976372 30388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:09.977694 30388 hybrid_clock.cc:648] HybridClock initialized: now 1790455209977612 us; error 44 us; skew 500 ppm
I20260926 20:40:09.977975 30388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:09 dist-test-slave-4d5t krb5kdc[29878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455209, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:09.980003 30388 init.cc:377] Logged in from keytab as oryx/127.23.44.253@KRBTEST.COM (short username oryx)
I20260926 20:40:09.981683 30388 webserver.cc:460] Webserver started at http://127.23.44.253:34749/ using document root <none> and password file <none>
I20260926 20:40:09.982054 30388 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:09.982158 30388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:09.984727 30388 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 20:40:09.986534 30508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:09.987011 30388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260926 20:40:09.987176 30388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/wal
uuid: "a6a97c669fd2448c8c74d85265130e1e"
format_stamp: "Formatted at 2026-09-26 20:40:09 on dist-test-slave-4d5t"
server_key: "09c29719423dcefe0cc6aa53bb606e84"
server_key_iv: "39e770cb555669dfca65ddafc7623585"
server_key_version: "encryptionkey@0"
I20260926 20:40:09.987344 30388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:09.999266 30388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:10.004112 30388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:10.039373 30388 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.253:40781
I20260926 20:40:10.039398 30559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.253:40781 every 8 connection(s)
I20260926 20:40:10.040340 30388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-1/data/info.pb
I20260926 20:40:10.047487 30560 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:40:10.050925 30560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: Bootstrap starting.
I20260926 20:40:10.064940 30560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:40:10.065441 30560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: Bootstrap complete.
I20260926 20:40:10.066310 30560 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true } }
I20260926 20:40:10.066543 30560 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a6a97c669fd2448c8c74d85265130e1e, State: Initialized, Role: LEARNER
I20260926 20:40:10.066793 30560 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true } }
I20260926 20:40:10.067619 30563 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true } } }
I20260926 20:40:10.067955 30560 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:40:10.067993 30563 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: This master's current role is: LEARNER
I20260926 20:40:10.080017 30574 catalog_manager.cc:1259] Loaded cluster ID: 7d81cd72aff04fd7b14a2d74ebb5d49b
I20260926 20:40:10.080147 30574 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: loading cluster ID for follower catalog manager: success
I20260926 20:40:10.082955 30574 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: acquiring CA information for follower catalog manager: success
I20260926 20:40:10.084977 30574 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 20:40:10.195457 30382 heartbeater.cc:503] Master 127.23.44.254:41983 was elected leader, sending a full tablet report...
W20260926 20:40:10.340938 30489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:10.341243 30489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:10.341374 30489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:10.350867 30489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:40:10.350994 30489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:10.351095 30489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:10.351625 30489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:40:10.351704 30489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:40:10.365113 30489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:41983,127.23.44.253:40781,127.23.44.252:34937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.252.keytab
--principal=oryx/127.23.44.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.252:34937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.44.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:10.366623 30489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:10.368156 30489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:10.384037 30580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:10.384047 30579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:10.385164 30489 server_base.cc:1034] running on GCE node
W20260926 20:40:10.388849 30582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:10.390161 30489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:10.391911 30489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:10.393196 30489 hybrid_clock.cc:648] HybridClock initialized: now 1790455210393139 us; error 52 us; skew 500 ppm
I20260926 20:40:10.394018 30489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:10 dist-test-slave-4d5t krb5kdc[29878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455210, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:10.398258 30489 init.cc:377] Logged in from keytab as oryx/127.23.44.252@KRBTEST.COM (short username oryx)
I20260926 20:40:10.400290 30489 webserver.cc:460] Webserver started at http://127.23.44.252:44769/ using document root <none> and password file <none>
I20260926 20:40:10.401306 30489 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:10.401449 30489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:10.402753 30489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:10.406483 30489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/data/instance:
uuid: "608b462fd1104559963abb6596143116"
format_stamp: "Formatted at 2026-09-26 20:40:10 on dist-test-slave-4d5t"
server_key: "138fe5ca6412ed3e2464182e2bd75bbb"
server_key_iv: "860a84ebafcb0a01e5adfaf26022e43a"
server_key_version: "encryptionkey@0"
I20260926 20:40:10.407382 30489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/wal/instance:
uuid: "608b462fd1104559963abb6596143116"
format_stamp: "Formatted at 2026-09-26 20:40:10 on dist-test-slave-4d5t"
server_key: "138fe5ca6412ed3e2464182e2bd75bbb"
server_key_iv: "860a84ebafcb0a01e5adfaf26022e43a"
server_key_version: "encryptionkey@0"
I20260926 20:40:10.413401 30489 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260926 20:40:10.417704 30589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:10.419256 30489 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:40:10.419469 30489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/wal
uuid: "608b462fd1104559963abb6596143116"
format_stamp: "Formatted at 2026-09-26 20:40:10 on dist-test-slave-4d5t"
server_key: "138fe5ca6412ed3e2464182e2bd75bbb"
server_key_iv: "860a84ebafcb0a01e5adfaf26022e43a"
server_key_version: "encryptionkey@0"
I20260926 20:40:10.419694 30489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:10.444018 30489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:10.450006 30489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:10.450349 30489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:10.475464 30489 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.252:34937
I20260926 20:40:10.475497 30640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.252:34937 every 8 connection(s)
I20260926 20:40:10.476940 30489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/data/info.pb
I20260926 20:40:10.481163 30641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:10.482937 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 30489
I20260926 20:40:10.493039 30641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:40:10 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.44.252@KRBTEST.COM
Sep 26 20:40:10 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455210, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.252@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
I20260926 20:40:10.517254 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.485733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37630 (local address 127.23.44.252:34937)
0926 20:40:10.487066 (+ 1333us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:10.487084 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:10.487152 (+ 68us) server_negotiation.cc:407] Connection header received
0926 20:40:10.489473 (+ 2321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:10.489501 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:10.489972 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:10.490345 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:10.492579 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.494618 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.497164 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.498880 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.499715 (+ 835us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.503971 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:10.503982 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:10.503997 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:10.504050 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:10.510451 (+ 6401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.511525 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.511535 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.511545 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.511758 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.512301 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.512306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.512309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.513232 (+ 923us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:10.514699 (+ 1467us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:10.515049 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 20:40:10.515563 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1012,"thread_start_us":636,"threads_started":1}
I20260926 20:40:10.520157 30646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.497344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:49640)
0926 20:40:10.498848 (+ 1504us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:10.498889 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:10.499103 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:10.499514 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:10.499523 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:10.500447 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:10.500847 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.500857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.501961 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.501968 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.503563 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.503579 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.504689 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.504694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.504914 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.505589 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:10.505616 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:10.511805 (+ 6189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:10.516612 (+ 4807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.516621 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.516629 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.517288 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.518022 (+ 734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.518026 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.518028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.518178 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.518865 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:10.518873 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:10.519059 (+ 186us) client_negotiation.cc:769] Sending connection context
0926 20:40:10.519314 (+ 255us) client_negotiation.cc:240] Negotiation successful
0926 20:40:10.519722 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1273,"thread_start_us":1099,"threads_started":1}
I20260926 20:40:10.521329 30645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.497418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.23.44.254:41983)
0926 20:40:10.498390 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:10.498398 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:10.498997 (+ 599us) server_negotiation.cc:407] Connection header received
0926 20:40:10.499238 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:10.499244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:10.499323 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:10.499449 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:10.501003 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.501805 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.503806 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.504456 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.504621 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.512079 (+ 7458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:10.512087 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:10.512093 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:10.512139 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:10.516355 (+ 4216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.517704 (+ 1349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.517711 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.517714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.517870 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.518319 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.518324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.518328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.518701 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:10.518886 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:10.520720 (+ 1834us) server_negotiation.cc:299] Negotiation successful
0926 20:40:10.520937 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":299,"threads_started":1}
I20260926 20:40:10.526648 30641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:40:10.532389 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.507192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:40781 (local address 127.0.0.1:54694)
0926 20:40:10.524792 (+ 17600us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:10.524814 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:10.524945 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:10.525395 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:10.525399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:10.525425 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:10.525743 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.525758 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.527235 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.527252 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.529822 (+ 2570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.529837 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.531331 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.531337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.531582 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.531671 (+ 89us) client_negotiation.cc:769] Sending connection context
0926 20:40:10.531785 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 20:40:10.531927 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17435,"thread_start_us":17802,"threads_started":1}
I20260926 20:40:10.533196 30649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.507494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.23.44.253:40781)
0926 20:40:10.524992 (+ 17498us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:10.524998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:10.525027 (+ 29us) server_negotiation.cc:407] Connection header received
0926 20:40:10.525094 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:10.525098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:10.525169 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:10.525352 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:10.525905 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.527003 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.530035 (+ 3032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.531144 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.531253 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.531533 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:10.532695 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0926 20:40:10.532835 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17351,"thread_start_us":17228,"threads_started":1}
I20260926 20:40:10.535348 30535 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260926 20:40:10.536382 30563 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c12f5c6e8bf842719909b3f57df1bbf0. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true } } }
I20260926 20:40:10.536715 30563 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: This master's current role is: LEARNER
Sep 26 20:40:10 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455210, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.252@KRBTEST.COM for oryx/127.23.44.253@KRBTEST.COM
I20260926 20:40:10.544093 30646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.527793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:40781 (local address 127.0.0.1:54704)
0926 20:40:10.528153 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:10.528172 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:10.528271 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:10.529089 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:10.529092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:10.529568 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:10.529823 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.529832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.531523 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.531529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.532820 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.532833 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.533818 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.533824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.534640 (+ 816us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.535384 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:10.535544 (+ 160us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:10.538483 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:10.541632 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.541638 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.541642 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.542050 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.542448 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.542453 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.542456 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.542598 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.543121 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:10.543126 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:10.543284 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 20:40:10.543432 (+ 148us) client_negotiation.cc:240] Negotiation successful
0926 20:40:10.543745 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187}
I20260926 20:40:10.544147 30650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.527795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54704 (local address 127.23.44.253:40781)
0926 20:40:10.528474 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:10.528516 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:10.528538 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:40:10.528656 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:10.528662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:10.528905 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:10.529029 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:10.529969 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.531341 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.532965 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.533644 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.533781 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.538743 (+ 4962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:10.538748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:10.538751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:10.538784 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:10.541477 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.542179 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.542185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.542192 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.542330 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.542713 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.542718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.542721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.543001 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:10.543157 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:10.543516 (+ 359us) server_negotiation.cc:299] Negotiation successful
0926 20:40:10.543745 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":495,"thread_start_us":308,"threads_started":1}
I20260926 20:40:10.545466 30641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:40:10 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455210, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.252@KRBTEST.COM for oryx/127.23.44.252@KRBTEST.COM
I20260926 20:40:10.559337 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.546300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37644 (local address 127.23.44.252:34937)
0926 20:40:10.546475 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:10.546479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:10.546616 (+ 137us) server_negotiation.cc:407] Connection header received
0926 20:40:10.546780 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:10.546784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:10.546841 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:10.546922 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:10.547702 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.548405 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.549607 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.550021 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.550134 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.553601 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:10.553605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:10.553608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:10.553635 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:10.556399 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.557410 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.557415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.557417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.557524 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.557917 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.557920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.557922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.558211 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:10.558319 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:10.558707 (+ 388us) server_negotiation.cc:299] Negotiation successful
0926 20:40:10.559055 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":159,"server-negotiator.queue_time_us":66}
I20260926 20:40:10.559340 30646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.546320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:34937 (local address 127.0.0.1:37644)
0926 20:40:10.546543 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:10.546558 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:10.546652 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:10.547009 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:10.547013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:10.547388 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:10.547552 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.547561 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.548539 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.548543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.549453 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.549464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.550225 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.550231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.550410 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.551123 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:10.551143 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:10.553408 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:10.556611 (+ 3203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.556619 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.556625 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.557215 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.557639 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.557642 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.557644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.557804 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.558344 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:10.558348 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:10.558483 (+ 135us) client_negotiation.cc:769] Sending connection context
0926 20:40:10.558611 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 20:40:10.558933 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 20:40:10.565055 30641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: Bootstrap starting.
I20260926 20:40:10.568361 30641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:10.569585 30641 log.cc:824] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:10.572451 30641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: No bootstrap required, opened a new log
I20260926 20:40:10.581135 30641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } }
I20260926 20:40:10.581544 30641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:10.581715 30641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 608b462fd1104559963abb6596143116, State: Initialized, Role: FOLLOWER
I20260926 20:40:10.582366 30641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } }
I20260926 20:40:10.583292 30652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } } }
I20260926 20:40:10.583750 30652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:40:10.584481 30641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 20:40:10.595453 30663 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:40:10.595638 30663 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:40:10.600698 30489 master_runner.cc:428] Detected that this master 608b462fd1104559963abb6596143116 is joining an existing cluster
I20260926 20:40:10.600826 30489 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.44.252:34937
I20260926 20:40:10.626471 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 29985
I20260926 20:40:10.631639 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.606162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37646 (local address 127.23.44.252:34937)
0926 20:40:10.606905 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:10.606914 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:10.610083 (+ 3169us) server_negotiation.cc:407] Connection header received
0926 20:40:10.610230 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:10.610236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:10.610332 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:10.610459 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:10.612282 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.613656 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.617420 (+ 3764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.618232 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.618330 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.621841 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:10.621847 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:10.621851 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:10.621884 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:10.627834 (+ 5950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.628795 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.628805 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.628809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.629013 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.629565 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.629573 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.629577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.630058 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:10.630231 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:10.630916 (+ 685us) server_negotiation.cc:299] Negotiation successful
0926 20:40:10.631198 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":257}
I20260926 20:40:10.632247 30670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.606162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:34937 (local address 127.0.0.1:37646)
0926 20:40:10.609970 (+ 3808us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:10.609987 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:10.610101 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:10.610636 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:10.610641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:10.611692 (+ 1051us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:10.612119 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.612133 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.614313 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.614318 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.617287 (+ 2969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.617300 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.618649 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.618656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.619005 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.619944 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:10.620027 (+ 83us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:10.621619 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:10.628001 (+ 6382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.628053 (+ 52us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.628058 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.628663 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.629269 (+ 606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.629277 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.629284 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.629445 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.630326 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:10.630334 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:10.630554 (+ 220us) client_negotiation.cc:769] Sending connection context
0926 20:40:10.631261 (+ 707us) client_negotiation.cc:240] Negotiation successful
0926 20:40:10.631604 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3654,"mutex_wait_us":468,"spinlock_wait_cycles":30592,"thread_start_us":3463,"threads_started":1}
I20260926 20:40:10.634182 30650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.606715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.23.44.253:40781)
0926 20:40:10.607010 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:10.607020 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:10.615887 (+ 8867us) server_negotiation.cc:407] Connection header received
0926 20:40:10.616200 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:10.616207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:10.616302 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:10.616384 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:10.618103 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.619299 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.621579 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.622328 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.622433 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.625931 (+ 3498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:10.625940 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:10.625947 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:10.626006 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:10.629848 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.630770 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.630777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.630780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.630942 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.632290 (+ 1348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.632297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.632300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.632715 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:10.632853 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:10.633538 (+ 685us) server_negotiation.cc:299] Negotiation successful
0926 20:40:10.633793 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260926 20:40:10.635461 30671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.608011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:40781 (local address 127.0.0.1:54718)
0926 20:40:10.615765 (+ 7754us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:10.615787 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:10.616010 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:10.616631 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:10.616637 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:10.617495 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:10.617886 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.617898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.619497 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.619501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.621397 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.621416 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.622849 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.622856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.623046 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.624107 (+ 1061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:10.624134 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:10.625691 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:10.630048 (+ 4357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.630053 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.630057 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.630611 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.631188 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.631193 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.631197 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.632039 (+ 842us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.632937 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:10.632941 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:10.633181 (+ 240us) client_negotiation.cc:769] Sending connection context
0926 20:40:10.634556 (+ 1375us) client_negotiation.cc:240] Negotiation successful
0926 20:40:10.634923 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7585,"thread_start_us":7391,"threads_started":1}
I20260926 20:40:10.638602 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.606073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:49642)
0926 20:40:10.609160 (+ 3087us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:10.609183 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:10.609377 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:10.610021 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:10.610028 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:10.610977 (+ 949us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:10.611934 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.611959 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.614262 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.614271 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.616999 (+ 2728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.617016 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.618317 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.618323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.619002 (+ 679us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.620092 (+ 1090us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:10.620114 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:10.621655 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:10.632958 (+ 11303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.632974 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.632983 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.633487 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.634765 (+ 1278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.634774 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.634781 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.635065 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.636193 (+ 1128us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:10.636201 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:10.636514 (+ 313us) client_negotiation.cc:769] Sending connection context
0926 20:40:10.636746 (+ 232us) client_negotiation.cc:240] Negotiation successful
0926 20:40:10.637399 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2930,"mutex_wait_us":460,"thread_start_us":2750,"threads_started":1}
I20260926 20:40:10.640095 30645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.605879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.23.44.254:41983)
0926 20:40:10.606193 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:10.606201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:10.609302 (+ 3101us) server_negotiation.cc:407] Connection header received
0926 20:40:10.609552 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:10.609579 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:10.609693 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:10.609780 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:10.612248 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.613933 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.617205 (+ 3272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.617914 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.618057 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.621875 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:10.621882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:10.621887 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:10.621930 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:10.631298 (+ 9368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.634140 (+ 2842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.634149 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.634157 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.634576 (+ 419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.635231 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.635240 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.635246 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.635900 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:10.636111 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:10.639160 (+ 3049us) server_negotiation.cc:299] Negotiation successful
0926 20:40:10.639552 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 20:40:10 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.44.193@KRBTEST.COM
Sep 26 20:40:10 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455207, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.254@KRBTEST.COM for oryx/127.23.44.252@KRBTEST.COM
I20260926 20:40:10.660907 30672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.628519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57462 (local address 127.23.44.193:35693)
0926 20:40:10.641672 (+ 13153us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:10.641680 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:10.641731 (+ 51us) server_negotiation.cc:407] Connection header received
0926 20:40:10.641792 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:10.641798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:10.642044 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:10.642468 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:10.644000 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.645545 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.647551 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.648491 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.648667 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.654461 (+ 5794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:10.654481 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:10.654503 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:10.654547 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:10.657871 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.658576 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.658587 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.658596 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.658725 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.659101 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.659107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.659110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.659460 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:10.659768 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:10.660133 (+ 365us) server_negotiation.cc:299] Negotiation successful
0926 20:40:10.660398 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12935,"thread_start_us":14138,"threads_started":1}
I20260926 20:40:10.662768 30108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:10.668748 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.646382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:34937 (local address 127.0.0.1:37648)
0926 20:40:10.646661 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:10.646679 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:10.646792 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:10.647213 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:10.647220 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:10.647632 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:10.647759 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.647767 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.648808 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.648813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.650585 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.650603 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.651840 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.651847 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.652115 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.652823 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:10.652848 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:10.655309 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:10.660262 (+ 4953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.660269 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.660274 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.660874 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.661299 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.661304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.661308 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.661467 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.667730 (+ 6263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:10.667736 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:10.667917 (+ 181us) client_negotiation.cc:769] Sending connection context
0926 20:40:10.668121 (+ 204us) client_negotiation.cc:240] Negotiation successful
0926 20:40:10.668362 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260926 20:40:10.669951 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.646445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37648 (local address 127.23.44.252:34937)
0926 20:40:10.646642 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:10.646653 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:10.646730 (+ 77us) server_negotiation.cc:407] Connection header received
0926 20:40:10.646978 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:10.646982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:10.647076 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:10.647187 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:10.647908 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.648640 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.650791 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.651655 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.651804 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.655514 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:10.655521 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:10.655526 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:10.655564 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:10.660043 (+ 4479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.661016 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.661024 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.661026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.661191 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.662013 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.662021 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.662026 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.667572 (+ 5546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:10.667771 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:10.669068 (+ 1297us) server_negotiation.cc:299] Negotiation successful
0926 20:40:10.669564 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 20:40:10.671974 29923 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.44.252:34937: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "608b462fd1104559963abb6596143116"
member_type: NON_VOTER
last_known_addr {
host: "127.23.44.252"
port: 34937
}
attrs {
promote: true
}
}
dest_uuid: "c12f5c6e8bf842719909b3f57df1bbf0"
cas_config_opid_index: 5
I20260926 20:40:10.673204 29923 consensus_queue.cc:237] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } }
I20260926 20:40:10.675169 30647 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 608b462fd1104559963abb6596143116 (127.23.44.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } } }
I20260926 20:40:10.676133 30616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:10.680626 30616 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 1 FOLLOWER]: Refusing update from remote peer c12f5c6e8bf842719909b3f57df1bbf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 20:40:10.692269 30535 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 LEARNER]: Refusing update from remote peer c12f5c6e8bf842719909b3f57df1bbf0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 20:40:10.698879 30647 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.44.252:34937
I20260926 20:40:10.699992 30489 master.cc:537] Master@127.23.44.252:34937 shutting down...
I20260926 20:40:10.704005 30677 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:10.706647 30675 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 20:40:10.717263 30535 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 608b462fd1104559963abb6596143116 (127.23.44.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } } }
I20260926 20:40:10.724267 30563 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } } }
I20260926 20:40:10.724659 30563 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: This master's current role is: LEARNER
I20260926 20:40:10.725244 30647 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0: attempting to promote NON_VOTER a6a97c669fd2448c8c74d85265130e1e to VOTER
I20260926 20:40:10.725342 30676 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: NON_VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: true } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } } }
I20260926 20:40:10.725785 30676 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:10.726187 30647 consensus_queue.cc:237] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } }
I20260926 20:40:10.728461 30535 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 LEARNER]: Refusing update from remote peer c12f5c6e8bf842719909b3f57df1bbf0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 20:40:10.729195 30674 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260926 20:40:10.731896 29908 consensus_peers.cc:487] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 -> Peer 608b462fd1104559963abb6596143116 (127.23.44.252:34937): Couldn't send request to peer 608b462fd1104559963abb6596143116. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 20:40:10.732537 30676 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a6a97c669fd2448c8c74d85265130e1e (127.23.44.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } } }
I20260926 20:40:10.733470 30535 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a6a97c669fd2448c8c74d85265130e1e (127.23.44.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } } }
I20260926 20:40:10.736822 30563 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } } }
I20260926 20:40:10.737109 30563 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:40:10.737634 30674 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } } }
I20260926 20:40:10.737761 30676 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } } }
I20260926 20:40:10.738035 30674 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:10.738078 30676 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: This master's current role is: LEADER
W20260926 20:40:10.740937 30595 tablet.cc:2369] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:40:10.743631 30489 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:10.744042 30489 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:10.744262 30489 tablet_replica.cc:330] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: stopping tablet replica
I20260926 20:40:10.750614 30489 master.cc:552] Master@127.23.44.252:34937 shutdown complete.
I20260926 20:40:10.750734 30489 master_runner.cc:305] Clearing existing system tablet
I20260926 20:40:10.751080 30489 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:10.751210 30489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:10.754808 30489 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 20:40:10.756422 30682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:10.756845 30489 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:10.757223 30489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 20:40:10.757335 30489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/wal
uuid: "608b462fd1104559963abb6596143116"
format_stamp: "Formatted at 2026-09-26 20:40:10 on dist-test-slave-4d5t"
server_key: "138fe5ca6412ed3e2464182e2bd75bbb"
server_key_iv: "860a84ebafcb0a01e5adfaf26022e43a"
server_key_version: "encryptionkey@0"
I20260926 20:40:10.761281 30489 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:10.766541 30489 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 20:40:10.766741 30489 log.cc:1196] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 20:40:10.767200 30489 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: Deleting consensus metadata
I20260926 20:40:10.767473 30489 master_runner.cc:315] Copying system tablet from 127.23.44.254:41983
I20260926 20:40:10.770398 30489 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: tablet copy: Beginning tablet copy session from remote peer at address 127.23.44.254:41983
I20260926 20:40:10.785101 30645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.771556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49656 (local address 127.23.44.254:41983)
0926 20:40:10.771804 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:10.771811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:10.771894 (+ 83us) server_negotiation.cc:407] Connection header received
0926 20:40:10.772180 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:10.772186 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:10.772268 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:10.772389 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:10.773796 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.774762 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.776246 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.776834 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.776976 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.779042 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:10.779049 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:10.779056 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:10.779094 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:10.782024 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.782755 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.782764 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.782768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.782964 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:10.783389 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:10.783396 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:10.783400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:10.783753 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:10.783912 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:10.784376 (+ 464us) server_negotiation.cc:299] Negotiation successful
0926 20:40:10.784680 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260926 20:40:10.785176 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:10.771239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:49656)
0926 20:40:10.771776 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:10.771799 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:10.772020 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:10.772411 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:10.772416 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:10.772964 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:10.773638 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.773650 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.774931 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.774941 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.776065 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:10.776077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:10.777002 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:10.777009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:10.777225 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:10.777966 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:10.777984 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:10.778819 (+ 835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:10.782187 (+ 3368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.782193 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.782198 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.782619 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.783091 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:10.783097 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:10.783101 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:10.783269 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:10.783900 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:10.783905 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:10.784080 (+ 175us) client_negotiation.cc:769] Sending connection context
0926 20:40:10.784264 (+ 184us) client_negotiation.cc:240] Negotiation successful
0926 20:40:10.784649 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":585,"threads_started":1}
I20260926 20:40:10.786171 29943 tablet_copy_service.cc:140] P c12f5c6e8bf842719909b3f57df1bbf0: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 608b462fd1104559963abb6596143116 ({username='oryx', principal='oryx/127.23.44.252@KRBTEST.COM'} at 127.0.0.1:49656)
I20260926 20:40:10.786351 29943 tablet_copy_service.cc:161] P c12f5c6e8bf842719909b3f57df1bbf0: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 608b462fd1104559963abb6596143116 at {username='oryx', principal='oryx/127.23.44.252@KRBTEST.COM'} at 127.0.0.1:49656: session id = 608b462fd1104559963abb6596143116-00000000000000000000000000000000
I20260926 20:40:10.789505 29943 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 20:40:10.791198 30489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 20:40:10.797081 30489 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: tablet copy: Starting download of 0 data blocks...
I20260926 20:40:10.797430 30489 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: tablet copy: Starting download of 1 WAL segments...
I20260926 20:40:10.800107 30489 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 20:40:10.804631 29943 tablet_copy_service.cc:342] P c12f5c6e8bf842719909b3f57df1bbf0: Request end of tablet copy session 608b462fd1104559963abb6596143116-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.44.252@KRBTEST.COM'} at 127.0.0.1:49656
I20260926 20:40:10.804798 29943 tablet_copy_service.cc:433] P c12f5c6e8bf842719909b3f57df1bbf0: ending tablet copy session 608b462fd1104559963abb6596143116-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 608b462fd1104559963abb6596143116
I20260926 20:40:10.815598 30489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:10.818420 30689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:10.822415 30690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:10.837044 30692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:10.837522 30489 server_base.cc:1034] running on GCE node
I20260926 20:40:10.837841 30489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:10.838512 30489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:10.839722 30489 hybrid_clock.cc:648] HybridClock initialized: now 1790455210839658 us; error 50 us; skew 500 ppm
I20260926 20:40:10.839965 30489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:10 dist-test-slave-4d5t krb5kdc[29878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455210, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:10.841974 30489 init.cc:377] Logged in from keytab as oryx/127.23.44.252@KRBTEST.COM (short username oryx)
I20260926 20:40:10.843441 30489 webserver.cc:460] Webserver started at http://127.23.44.252:45959/ using document root <none> and password file <none>
I20260926 20:40:10.843762 30489 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:10.843875 30489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:10.846632 30489 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260926 20:40:10.848307 30699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:10.848795 30489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260926 20:40:10.848929 30489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/wal
uuid: "608b462fd1104559963abb6596143116"
format_stamp: "Formatted at 2026-09-26 20:40:10 on dist-test-slave-4d5t"
server_key: "138fe5ca6412ed3e2464182e2bd75bbb"
server_key_iv: "860a84ebafcb0a01e5adfaf26022e43a"
server_key_version: "encryptionkey@0"
I20260926 20:40:10.849123 30489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:10.857841 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 29985
I20260926 20:40:10.861378 30489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:10.865909 30489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:10.886701 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:35693
--local_ip_for_outbound_sockets=127.23.44.193
--tserver_master_addrs=127.23.44.254:41983
--webserver_port=41785
--webserver_interface=127.23.44.193
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab
--principal=oryx/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.44.254:41983,127.23.44.253:40781
--tserver_master_addrs=127.23.44.254:41983,127.23.44.253:40781,127.23.44.252:34937 with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:40:10.900008 30489 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.252:34937
I20260926 20:40:10.900051 30751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.252:34937 every 8 connection(s)
I20260926 20:40:10.901309 30489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/master-2/data/info.pb
I20260926 20:40:10.906618 30752 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:40:10.910672 30752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: Bootstrap starting.
I20260926 20:40:10.929260 30752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:40:10.929754 30752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: Bootstrap complete.
I20260926 20:40:10.930795 30752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } }
I20260926 20:40:10.931069 30752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 608b462fd1104559963abb6596143116, State: Initialized, Role: LEARNER
I20260926 20:40:10.931427 30752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } }
I20260926 20:40:10.932813 30752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:40:10.932641 30755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } } }
I20260926 20:40:10.933491 30755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: This master's current role is: LEARNER
I20260926 20:40:10.947834 30766 catalog_manager.cc:1259] Loaded cluster ID: 7d81cd72aff04fd7b14a2d74ebb5d49b
I20260926 20:40:10.947962 30766 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: loading cluster ID for follower catalog manager: success
I20260926 20:40:10.951716 30766 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: acquiring CA information for follower catalog manager: success
I20260926 20:40:10.954458 30766 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 20:40:11.264802 30767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:11.257144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:34937 (local address 127.0.0.1:37662)
0926 20:40:11.257719 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:11.257747 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:11.257897 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:11.258365 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:11.258369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:11.258391 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:11.258586 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:11.258594 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.260491 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.260495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:11.261812 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:11.261825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.264101 (+ 2276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.264109 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:11.264278 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:11.264306 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 20:40:11.264395 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 20:40:11.264503 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":266,"threads_started":1}
I20260926 20:40:11.265169 30768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:11.257325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.23.44.252:34937)
0926 20:40:11.257980 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:11.257987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:11.258009 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:40:11.258083 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:11.258089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:11.258186 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:11.258325 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:11.258732 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.260319 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.261996 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.263838 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.264030 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:11.264256 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:11.264452 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 20:40:11.264639 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":387,"threads_started":1}
I20260926 20:40:11.266141 30726 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260926 20:40:11.266988 30755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c12f5c6e8bf842719909b3f57df1bbf0. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } } }
I20260926 20:40:11.267444 30755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: This master's current role is: LEARNER
W20260926 20:40:11.272346 30746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:11.272709 30746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:11.272876 30746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:11.281165 30746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:11.281358 30746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:11.281548 30746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:40:11.292371 30746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab
--principal=oryx/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:35693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=41785
--tserver_master_addrs=127.23.44.254:41983,127.23.44.253:40781,127.23.44.252:34937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:11.293823 30746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:11.295150 30746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:11.310954 30774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:11.310904 30776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:11.311472 30773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:11.312371 30746 server_base.cc:1034] running on GCE node
I20260926 20:40:11.313184 30746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:11.314291 30746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:11.315589 30746 hybrid_clock.cc:648] HybridClock initialized: now 1790455211315395 us; error 126 us; skew 500 ppm
I20260926 20:40:11.315938 30746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:11 dist-test-slave-4d5t krb5kdc[29878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455211, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:11.320133 30746 init.cc:377] Logged in from keytab as oryx/127.23.44.193@KRBTEST.COM (short username oryx)
I20260926 20:40:11.322032 30746 webserver.cc:460] Webserver started at http://127.23.44.193:41785/ using document root <none> and password file <none>
I20260926 20:40:11.322808 30746 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:11.322978 30746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:11.329123 30746 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 20:40:11.332640 30783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:11.334905 30746 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:40:11.335156 30746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "a32161d75a4447f7a69c09e3024251f1"
format_stamp: "Formatted at 2026-09-26 20:40:07 on dist-test-slave-4d5t"
server_key: "2c3ed07deb92edb758a25c0ec5d9e9b0"
server_key_iv: "298ddd82470be1216b99449bac1a3771"
server_key_version: "encryptionkey@0"
I20260926 20:40:11.336414 30746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:11.369953 30746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:11.375181 30746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:11.375511 30746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:11.376797 30746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:11.378499 30746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:11.378634 30746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:11.378778 30746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:11.379289 30746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:11.423303 30746 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:35693
I20260926 20:40:11.423393 30895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:35693 every 8 connection(s)
I20260926 20:40:11.424770 30746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:40:11.429095 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 30746
I20260926 20:40:11.429473 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 30119
Sep 26 20:40:11 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455211, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
Sep 26 20:40:11 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455211, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.253@KRBTEST.COM
Sep 26 20:40:11 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455211, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.252@KRBTEST.COM
Sep 26 20:40:11 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.44.194@KRBTEST.COM
I20260926 20:40:11.470947 30768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:11.428526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:47251 (local address 127.23.44.252:34937)
0926 20:40:11.428755 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:11.428760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:11.447521 (+ 18761us) server_negotiation.cc:407] Connection header received
0926 20:40:11.447798 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:11.447805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:11.447908 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:11.447976 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:11.451270 (+ 3294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.452336 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.455395 (+ 3059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.456067 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.456174 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:11.462554 (+ 6380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:11.462561 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:11.462566 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:11.462609 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:11.466382 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:11.467738 (+ 1356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:11.467744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:11.467749 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:11.467905 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:11.468532 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:11.468539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:11.468542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:11.468937 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:11.469065 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:11.470260 (+ 1195us) server_negotiation.cc:299] Negotiation successful
0926 20:40:11.470552 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260926 20:40:11.471045 30900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:11.427807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:47481 (local address 127.23.44.254:41983)
0926 20:40:11.428712 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:11.428719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:11.447279 (+ 18560us) server_negotiation.cc:407] Connection header received
0926 20:40:11.447361 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:11.447369 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:11.447465 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:11.447560 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:11.451753 (+ 4193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.452835 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.455406 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.456062 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.456177 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:11.461970 (+ 5793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:11.461977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:11.461983 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:11.462030 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:11.466540 (+ 4510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:11.467729 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:11.467735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:11.467740 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:11.467926 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:11.468518 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:11.468523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:11.468526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:11.468947 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:11.469200 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:11.470478 (+ 1278us) server_negotiation.cc:299] Negotiation successful
0926 20:40:11.470702 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":938,"threads_started":1}
I20260926 20:40:11.472312 30901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:11.428591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:40781 (local address 127.23.44.193:40377)
0926 20:40:11.436266 (+ 7675us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:11.436366 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:11.440119 (+ 3753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:11.449394 (+ 9275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:11.449398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:11.450160 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:11.451040 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:11.451077 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.453221 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.453228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:11.455232 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:11.455246 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.456477 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.456485 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:11.456994 (+ 509us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:11.458061 (+ 1067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:11.458089 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:11.462923 (+ 4834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:11.467375 (+ 4452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:11.467382 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:11.467388 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:11.467916 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:11.468379 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:11.468383 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:11.468386 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:11.468526 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:11.469233 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:11.469244 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:11.469788 (+ 544us) client_negotiation.cc:769] Sending connection context
0926 20:40:11.470119 (+ 331us) client_negotiation.cc:240] Negotiation successful
0926 20:40:11.470719 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"mutex_wait_us":315,"spinlock_wait_cycles":22272,"thread_start_us":928,"threads_started":1}
I20260926 20:40:11.472388 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:11.430465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51212 (local address 127.23.44.194:39483)
0926 20:40:11.447811 (+ 17346us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:11.447821 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:11.447853 (+ 32us) server_negotiation.cc:407] Connection header received
0926 20:40:11.447916 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:11.447925 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:11.448232 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:11.448416 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:11.450851 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.452959 (+ 2108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.454983 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.455859 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.455965 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:11.463301 (+ 7336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:11.463322 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:11.463343 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:11.463401 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:11.467916 (+ 4515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:11.468975 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:11.468988 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:11.468996 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:11.469308 (+ 312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:11.469810 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:11.469818 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:11.469824 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:11.470271 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:11.470560 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:11.471696 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0926 20:40:11.471968 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17145,"thread_start_us":16985,"threads_started":1}
I20260926 20:40:11.472993 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:11.430052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:34937 (local address 127.23.44.193:47251)
0926 20:40:11.447395 (+ 17343us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:11.447421 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:11.447627 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:11.448209 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:11.448213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:11.450203 (+ 1990us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:11.451043 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:11.451075 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.452530 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.452535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:11.455222 (+ 2687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:11.455236 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.456520 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.456525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:11.456994 (+ 469us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:11.457962 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:11.457997 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:11.462297 (+ 4300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:11.466573 (+ 4276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:11.466586 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:11.466612 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:11.467576 (+ 964us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:11.468062 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:11.468067 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:11.468070 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:11.468396 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:11.469306 (+ 910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:11.469320 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:11.469805 (+ 485us) client_negotiation.cc:769] Sending connection context
0926 20:40:11.471656 (+ 1851us) client_negotiation.cc:240] Negotiation successful
0926 20:40:11.471949 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17215,"mutex_wait_us":475,"spinlock_wait_cycles":129024,"thread_start_us":17061,"threads_started":1}
I20260926 20:40:11.474272 30243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:11.475389 30896 heartbeater.cc:347] Connected to a master server at 127.23.44.252:34937
I20260926 20:40:11.475768 30896 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:11.476642 30896 heartbeater.cc:511] Master 127.23.44.252:34937 requested a full tablet report, sending...
I20260926 20:40:11.476657 30902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:11.429289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:40377 (local address 127.23.44.253:40781)
0926 20:40:11.449003 (+ 19714us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:11.449010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:11.449034 (+ 24us) server_negotiation.cc:407] Connection header received
0926 20:40:11.449091 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:11.449097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:11.449199 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:11.449323 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:11.451288 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.453057 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.455407 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.456013 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.456129 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:11.463185 (+ 7056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:11.463193 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:11.463201 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:11.463263 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:11.467208 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:11.468056 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:11.468063 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:11.468067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:11.468240 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:11.468708 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:11.468712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:11.468715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:11.469042 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:11.469162 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:11.472875 (+ 3713us) server_negotiation.cc:299] Negotiation successful
0926 20:40:11.473092 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19574,"thread_start_us":19443,"threads_started":1}
I20260926 20:40:11.477583 30898 heartbeater.cc:347] Connected to a master server at 127.23.44.253:40781
I20260926 20:40:11.477726 30898 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:11.477789 30903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:11.429402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.193:47481)
0926 20:40:11.446898 (+ 17496us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:11.446925 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:11.447107 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:11.448102 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:11.448120 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:11.451257 (+ 3137us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:11.451555 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:11.451567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.453005 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.453015 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:11.455233 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:11.455242 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:11.456478 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:11.456487 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:11.456991 (+ 504us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:11.457962 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:11.458016 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:11.461677 (+ 3661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:11.466730 (+ 5053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:11.466738 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:11.466744 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:11.467562 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:11.468084 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:11.468088 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:11.468091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:11.468390 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:11.469248 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:11.469252 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:11.469788 (+ 536us) client_negotiation.cc:769] Sending connection context
0926 20:40:11.470124 (+ 336us) client_negotiation.cc:240] Negotiation successful
0926 20:40:11.477180 (+ 7056us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17306,"mutex_wait_us":6920,"spinlock_wait_cycles":37248,"thread_start_us":17040,"threads_started":1}
I20260926 20:40:11.478350 30898 heartbeater.cc:511] Master 127.23.44.253:40781 requested a full tablet report, sending...
I20260926 20:40:11.479599 30716 ts_manager.cc:194] Registered new tserver with Master: a32161d75a4447f7a69c09e3024251f1 (127.23.44.193:35693)
I20260926 20:40:11.480403 30525 ts_manager.cc:194] Registered new tserver with Master: a32161d75a4447f7a69c09e3024251f1 (127.23.44.193:35693)
I20260926 20:40:11.480777 30897 heartbeater.cc:347] Connected to a master server at 127.23.44.254:41983
I20260926 20:40:11.480919 30897 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:11.481463 30897 heartbeater.cc:511] Master 127.23.44.254:41983 requested a full tablet report, sending...
I20260926 20:40:11.482632 29923 ts_manager.cc:194] Re-registered known tserver with Master: a32161d75a4447f7a69c09e3024251f1 (127.23.44.193:35693)
I20260926 20:40:11.483456 29923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:47481
I20260926 20:40:11.563174 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 30119
I20260926 20:40:11.579856 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:39483
--local_ip_for_outbound_sockets=127.23.44.194
--tserver_master_addrs=127.23.44.254:41983
--webserver_port=36257
--webserver_interface=127.23.44.194
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab
--principal=oryx/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.44.254:41983,127.23.44.253:40781
--tserver_master_addrs=127.23.44.254:41983,127.23.44.253:40781,127.23.44.252:34937 with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:40:11.684981 30676 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: NON_VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: true } } }
I20260926 20:40:11.685442 30676 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:11.687906 30676 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0: attempting to promote NON_VOTER 608b462fd1104559963abb6596143116 to VOTER
I20260926 20:40:11.688534 30676 consensus_queue.cc:237] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } }
I20260926 20:40:11.690573 30726 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 1 LEARNER]: Refusing update from remote peer c12f5c6e8bf842719909b3f57df1bbf0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260926 20:40:11.690573 30535 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 FOLLOWER]: Refusing update from remote peer c12f5c6e8bf842719909b3f57df1bbf0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260926 20:40:11.691221 30910 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 20:40:11.691685 30909 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 20:40:11.695128 30535 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 608b462fd1104559963abb6596143116 (127.23.44.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } } }
I20260926 20:40:11.697438 30726 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 608b462fd1104559963abb6596143116 (127.23.44.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } } }
I20260926 20:40:11.694355 30910 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 608b462fd1104559963abb6596143116 (127.23.44.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } } }
I20260926 20:40:11.701071 30913 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } } }
I20260926 20:40:11.701525 30913 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:40:11.702049 30755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } } }
I20260926 20:40:11.702338 30755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:40:11.704869 30910 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } } }
I20260926 20:40:11.704803 30909 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } } }
I20260926 20:40:11.705109 30910 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:11.705199 30909 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:11.704955 30676 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } } }
I20260926 20:40:11.705364 30676 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0 [sys.catalog]: This master's current role is: LEADER
W20260926 20:40:11.945268 30908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:11.945571 30908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:11.945667 30908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:11.953102 30908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:11.953225 30908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:11.953339 30908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:40:11.963150 30908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab
--principal=oryx/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:39483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=36257
--tserver_master_addrs=127.23.44.254:41983,127.23.44.253:40781,127.23.44.252:34937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:11.964449 30908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:11.965828 30908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:11.981376 30919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:11.981423 30922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:11.981376 30920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:11.982286 30908 server_base.cc:1034] running on GCE node
I20260926 20:40:11.982995 30908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:11.984289 30908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:11.985632 30908 hybrid_clock.cc:648] HybridClock initialized: now 1790455211985581 us; error 92 us; skew 500 ppm
I20260926 20:40:11.986037 30908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:11 dist-test-slave-4d5t krb5kdc[29878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455211, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:11.990484 30908 init.cc:377] Logged in from keytab as oryx/127.23.44.194@KRBTEST.COM (short username oryx)
I20260926 20:40:11.992062 30908 webserver.cc:460] Webserver started at http://127.23.44.194:36257/ using document root <none> and password file <none>
I20260926 20:40:11.992889 30908 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:11.993055 30908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:11.998847 30908 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:40:12.002077 30929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:12.003832 30908 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:40:12.004029 30908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "a08386306b2d4fbc83729667f83d09fb"
format_stamp: "Formatted at 2026-09-26 20:40:08 on dist-test-slave-4d5t"
server_key: "aca9a01a9648c5037350ed4945f58b1f"
server_key_iv: "1ab9d38f8e24a7da57a3db24ac4e3cb5"
server_key_version: "encryptionkey@0"
I20260926 20:40:12.005123 30908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:12.022961 30908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:12.027810 30908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:12.028126 30908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:12.029435 30908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:12.030918 30908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:12.031077 30908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:12.031200 30908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:12.031297 30908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:12.071372 30908 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:39483
I20260926 20:40:12.071592 31041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:39483 every 8 connection(s)
I20260926 20:40:12.073191 30908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:40:12.077781 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 30908
I20260926 20:40:12.078099 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 30254
Sep 26 20:40:12 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455211, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.252@KRBTEST.COM
Sep 26 20:40:12 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455211, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.253@KRBTEST.COM
Sep 26 20:40:12 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455211, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
Sep 26 20:40:12 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.44.195@KRBTEST.COM
I20260926 20:40:12.126456 31046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.076849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:44077 (local address 127.23.44.253:40781)
0926 20:40:12.102182 (+ 25333us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:12.102187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:12.102203 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:40:12.102252 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:12.102255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:12.102321 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:12.102459 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:12.107340 (+ 4881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.108780 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.111119 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.111705 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.111824 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.118109 (+ 6285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:12.118116 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:12.118121 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:12.118171 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:12.121855 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.123416 (+ 1561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.123421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.123425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.123591 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.124120 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.124125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.124128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.124521 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:12.124678 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:12.125851 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0926 20:40:12.126079 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25219,"thread_start_us":25074,"threads_started":1}
I20260926 20:40:12.126770 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.078376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:36127 (local address 127.23.44.252:34937)
0926 20:40:12.096106 (+ 17730us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:12.096126 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:12.096172 (+ 46us) server_negotiation.cc:407] Connection header received
0926 20:40:12.096264 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:12.096273 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:12.096405 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:12.101179 (+ 4774us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:12.101422 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.103725 (+ 2303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.107733 (+ 4008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.108449 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.108659 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.114657 (+ 5998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:12.114667 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:12.114672 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:12.114718 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:12.119964 (+ 5246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.122875 (+ 2911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.122885 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.122893 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.123151 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.124020 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.124027 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.124031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.124492 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:12.124697 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:12.126074 (+ 1377us) server_negotiation.cc:299] Negotiation successful
0926 20:40:12.126354 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17422,"thread_start_us":17235,"threads_started":1}
I20260926 20:40:12.127666 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.077187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:36069 (local address 127.23.44.254:41983)
0926 20:40:12.101246 (+ 24059us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:12.101255 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:12.101283 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:40:12.101359 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:12.101367 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:12.101470 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:12.101617 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:12.107378 (+ 5761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.108458 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.112086 (+ 3628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.112840 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.112998 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.118958 (+ 5960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:12.118967 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:12.118973 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:12.119039 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:12.123205 (+ 4166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.124244 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.124250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.124253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.124411 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.124956 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.124961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.124964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.125309 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:12.125446 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:12.126893 (+ 1447us) server_negotiation.cc:299] Negotiation successful
0926 20:40:12.127161 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23881,"thread_start_us":23767,"threads_started":1}
I20260926 20:40:12.128330 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.079554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48198 (local address 127.23.44.195:33263)
0926 20:40:12.099768 (+ 20214us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:12.099780 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:12.099819 (+ 39us) server_negotiation.cc:407] Connection header received
0926 20:40:12.099904 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:12.099917 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:12.100253 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:12.100461 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:12.105190 (+ 4729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.107166 (+ 1976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.110729 (+ 3563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.111487 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.111628 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.118824 (+ 7196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:12.118848 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:12.118871 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:12.118914 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:12.123037 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.123932 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.123940 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.123944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.124113 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.124527 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.124531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.124534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.124975 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:12.125252 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:12.127058 (+ 1806us) server_negotiation.cc:299] Negotiation successful
0926 20:40:12.127333 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20020,"thread_start_us":19845,"threads_started":1}
I20260926 20:40:12.128885 31048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.077636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.194:36069)
0926 20:40:12.097708 (+ 20072us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:12.097738 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:12.097901 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:12.102678 (+ 4777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:12.102689 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:12.105126 (+ 2437us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:12.106338 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.106355 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.108634 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.108640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.110523 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.110539 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.113061 (+ 2522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.113067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.113269 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.114092 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:12.114120 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:12.118695 (+ 4575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:12.123393 (+ 4698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:12.123399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:12.123404 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:12.124090 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:12.124582 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:12.124590 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:12.124597 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:12.124814 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:12.125517 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:12.125522 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:12.125658 (+ 136us) client_negotiation.cc:769] Sending connection context
0926 20:40:12.125789 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 20:40:12.126369 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19820,"mutex_wait_us":38,"spinlock_wait_cycles":35072,"thread_start_us":10273,"threads_started":1}
I20260926 20:40:12.128955 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.080789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:40781 (local address 127.23.44.194:44077)
0926 20:40:12.098985 (+ 18196us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:12.099021 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:12.099161 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:12.102557 (+ 3396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:12.102563 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:12.104137 (+ 1574us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:12.107138 (+ 3001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.107154 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.108905 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.108909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.110943 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.110965 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.112373 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.112381 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.112622 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.113715 (+ 1093us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:12.113738 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:12.117385 (+ 3647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:12.122734 (+ 5349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:12.122740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:12.122744 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:12.123268 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:12.123733 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:12.123743 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:12.123746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:12.123973 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:12.124767 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:12.124777 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:12.125359 (+ 582us) client_negotiation.cc:769] Sending connection context
0926 20:40:12.126680 (+ 1321us) client_negotiation.cc:240] Negotiation successful
0926 20:40:12.127024 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18023,"mutex_wait_us":34,"spinlock_wait_cycles":5376,"thread_start_us":17854,"threads_started":1}
I20260926 20:40:12.128845 31049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.077635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:34937 (local address 127.23.44.194:36127)
0926 20:40:12.090300 (+ 12665us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:12.090395 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:12.094614 (+ 4219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:12.098498 (+ 3884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:12.098523 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:12.099706 (+ 1183us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:12.100911 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.100946 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.103982 (+ 3036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.103992 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.107561 (+ 3569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.107575 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.108901 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.108909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.109361 (+ 452us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.110425 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:12.110472 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:12.114312 (+ 3840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:12.120136 (+ 5824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:12.120149 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:12.120202 (+ 53us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:12.122588 (+ 2386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:12.123393 (+ 805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:12.123399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:12.123404 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:12.123838 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:12.124764 (+ 926us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:12.124783 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:12.125362 (+ 579us) client_negotiation.cc:769] Sending connection context
0926 20:40:12.125716 (+ 354us) client_negotiation.cc:240] Negotiation successful
0926 20:40:12.127524 (+ 1808us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10060,"mutex_wait_us":43,"spinlock_wait_cycles":65536,"thread_start_us":19451,"threads_started":1}
I20260926 20:40:12.130412 30377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:12.131011 31044 heartbeater.cc:347] Connected to a master server at 127.23.44.253:40781
I20260926 20:40:12.131111 31043 heartbeater.cc:347] Connected to a master server at 127.23.44.254:41983
I20260926 20:40:12.131422 31043 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:12.131474 31042 heartbeater.cc:347] Connected to a master server at 127.23.44.252:34937
I20260926 20:40:12.131609 31042 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:12.131429 31044 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:12.132818 31042 heartbeater.cc:511] Master 127.23.44.252:34937 requested a full tablet report, sending...
I20260926 20:40:12.132905 31044 heartbeater.cc:511] Master 127.23.44.253:40781 requested a full tablet report, sending...
I20260926 20:40:12.134166 31043 heartbeater.cc:511] Master 127.23.44.254:41983 requested a full tablet report, sending...
I20260926 20:40:12.134872 30716 ts_manager.cc:194] Registered new tserver with Master: a08386306b2d4fbc83729667f83d09fb (127.23.44.194:39483)
I20260926 20:40:12.135175 30525 ts_manager.cc:194] Registered new tserver with Master: a08386306b2d4fbc83729667f83d09fb (127.23.44.194:39483)
I20260926 20:40:12.136766 29923 ts_manager.cc:194] Re-registered known tserver with Master: a08386306b2d4fbc83729667f83d09fb (127.23.44.194:39483)
I20260926 20:40:12.138306 29923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:36069
I20260926 20:40:12.229588 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 30254
I20260926 20:40:12.247674 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:33263
--local_ip_for_outbound_sockets=127.23.44.195
--tserver_master_addrs=127.23.44.254:41983
--webserver_port=45361
--webserver_interface=127.23.44.195
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab
--principal=oryx/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.44.254:41983,127.23.44.253:40781
--tserver_master_addrs=127.23.44.254:41983,127.23.44.253:40781,127.23.44.252:34937 with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:40:12.485942 30897 heartbeater.cc:503] Master 127.23.44.254:41983 was elected leader, sending a full tablet report...
W20260926 20:40:12.617357 31055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:12.617672 31055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:12.617827 31055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:12.625195 31055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:12.625327 31055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:12.625528 31055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:40:12.636695 31055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab
--principal=oryx/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:33263
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=45361
--tserver_master_addrs=127.23.44.254:41983,127.23.44.253:40781,127.23.44.252:34937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:12.638051 31055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:12.639382 31055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:12.654817 31061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:12.654884 31063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:12.654911 31060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:12.656129 31055 server_base.cc:1034] running on GCE node
I20260926 20:40:12.657354 31055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:12.658830 31055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:12.660169 31055 hybrid_clock.cc:648] HybridClock initialized: now 1790455212660106 us; error 75 us; skew 500 ppm
I20260926 20:40:12.660712 31055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:12 dist-test-slave-4d5t krb5kdc[29878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455212, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:12.665059 31055 init.cc:377] Logged in from keytab as oryx/127.23.44.195@KRBTEST.COM (short username oryx)
I20260926 20:40:12.666827 31055 webserver.cc:460] Webserver started at http://127.23.44.195:45361/ using document root <none> and password file <none>
I20260926 20:40:12.667629 31055 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:12.667795 31055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:12.674141 31055 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:40:12.677626 31070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:12.679545 31055 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:40:12.679864 31055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "e01a9a75c042415dacc31c4cfb89a56b"
format_stamp: "Formatted at 2026-09-26 20:40:09 on dist-test-slave-4d5t"
server_key: "cecf02885febca73aa55352d763394c6"
server_key_iv: "adc83d6ceeca43c0d9c2e1bb9a0cbb36"
server_key_version: "encryptionkey@0"
I20260926 20:40:12.681437 31055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:12.696657 31055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:12.702623 31055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:12.703011 31055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:12.704331 31055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:12.706307 31055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:12.706476 31055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:12.706629 31055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:12.706715 31055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:12.745379 31055 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:33263
I20260926 20:40:12.745543 31182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:33263 every 8 connection(s)
I20260926 20:40:12.746915 31055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:40:12.752751 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 31055
Sep 26 20:40:12 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455212, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.252@KRBTEST.COM
Sep 26 20:40:12 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455212, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
Sep 26 20:40:12 dist-test-slave-4d5t krb5kdc[29878](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455212, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.253@KRBTEST.COM
I20260926 20:40:12.815907 31190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.751311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:60783 (local address 127.23.44.252:34937)
0926 20:40:12.781395 (+ 30084us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:12.781405 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:12.781440 (+ 35us) server_negotiation.cc:407] Connection header received
0926 20:40:12.789894 (+ 8454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:12.789946 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:12.790096 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:12.790215 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:12.792097 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.793761 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.797953 (+ 4192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.798888 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.799018 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.805383 (+ 6365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:12.805393 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:12.805401 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:12.805459 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:12.810499 (+ 5040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.811790 (+ 1291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.811799 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.811806 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.811985 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.812677 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.812685 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.812690 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.813112 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:12.813312 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:12.814947 (+ 1635us) server_negotiation.cc:299] Negotiation successful
0926 20:40:12.815286 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29891,"thread_start_us":29761,"threads_started":1}
I20260926 20:40:12.819000 31193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.750949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:34937 (local address 127.23.44.195:60783)
0926 20:40:12.776584 (+ 25635us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:12.776671 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:12.788897 (+ 12226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:12.790524 (+ 1627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:12.790533 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:12.791267 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:12.791800 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.791839 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.794072 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.794081 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.797694 (+ 3613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.797718 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.799451 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.799476 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.799997 (+ 521us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.801125 (+ 1128us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:12.801169 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:12.805009 (+ 3840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:12.810726 (+ 5717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:12.810743 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:12.810772 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:12.811608 (+ 836us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:12.812161 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:12.812169 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:12.812176 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:12.812504 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:12.813281 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:12.813295 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:12.814102 (+ 807us) client_negotiation.cc:769] Sending connection context
0926 20:40:12.816675 (+ 2573us) client_negotiation.cc:240] Negotiation successful
0926 20:40:12.817302 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25089,"mutex_wait_us":147,"thread_start_us":45520,"threads_started":1}
I20260926 20:40:12.821549 31183 heartbeater.cc:347] Connected to a master server at 127.23.44.252:34937
I20260926 20:40:12.821923 31183 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:12.822842 31183 heartbeater.cc:511] Master 127.23.44.252:34937 requested a full tablet report, sending...
I20260926 20:40:12.823194 31187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.750247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:33587 (local address 127.23.44.253:40781)
0926 20:40:12.775099 (+ 24852us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:12.775111 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:12.797189 (+ 22078us) server_negotiation.cc:407] Connection header received
0926 20:40:12.797455 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:12.797461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:12.797555 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:12.797671 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:12.799748 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.800984 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.803608 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.804326 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.804461 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.814646 (+ 10185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:12.814655 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:12.814661 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:12.814708 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:12.819067 (+ 4359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.820160 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.820170 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.820177 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.820371 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.820922 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.820929 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.820934 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.821327 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:12.821485 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:12.822284 (+ 799us) server_negotiation.cc:299] Negotiation successful
0926 20:40:12.822608 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24537,"thread_start_us":30536,"threads_started":1}
I20260926 20:40:12.823799 31191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.751801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.195:58799)
0926 20:40:12.782143 (+ 30342us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:12.782180 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:12.783780 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:12.785505 (+ 1725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:12.785526 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:12.787077 (+ 1551us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:12.792106 (+ 5029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.792123 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.794405 (+ 2282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.794414 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.799133 (+ 4719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.799157 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.801635 (+ 2478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.801645 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.803209 (+ 1564us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.804864 (+ 1655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:12.804899 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:12.808534 (+ 3635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:12.820419 (+ 11885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:12.820426 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:12.820430 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:12.821100 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:12.821597 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:12.821602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:12.821605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:12.821855 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:12.822648 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:12.822659 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:12.822806 (+ 147us) client_negotiation.cc:769] Sending connection context
0926 20:40:12.823050 (+ 244us) client_negotiation.cc:240] Negotiation successful
0926 20:40:12.823345 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30127,"mutex_wait_us":2697,"spinlock_wait_cycles":39296,"thread_start_us":29869,"threads_started":1}
I20260926 20:40:12.824913 30716 ts_manager.cc:194] Registered new tserver with Master: e01a9a75c042415dacc31c4cfb89a56b (127.23.44.195:33263)
I20260926 20:40:12.825268 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.750949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:40781 (local address 127.23.44.195:33587)
0926 20:40:12.797018 (+ 46069us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:12.797038 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:12.797298 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:12.797869 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:12.797874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:12.798578 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:12.799556 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.799571 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.801199 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.801204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.803041 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.803056 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.807053 (+ 3997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.807060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.807299 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.808816 (+ 1517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:12.808840 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:12.814352 (+ 5512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:12.819281 (+ 4929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:12.819291 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:12.819298 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:12.819983 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:12.820546 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:12.820584 (+ 38us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:12.820587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:12.820768 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:12.821606 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:12.821614 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:12.821897 (+ 283us) client_negotiation.cc:769] Sending connection context
0926 20:40:12.823809 (+ 1912us) client_negotiation.cc:240] Negotiation successful
0926 20:40:12.824532 (+ 723us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45857,"mutex_wait_us":5477,"spinlock_wait_cycles":34176,"thread_start_us":19917,"threads_started":1}
I20260926 20:40:12.826543 31185 heartbeater.cc:347] Connected to a master server at 127.23.44.253:40781
I20260926 20:40:12.826678 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.750551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:58799 (local address 127.23.44.254:41983)
0926 20:40:12.775587 (+ 25036us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:12.775597 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:12.782434 (+ 6837us) server_negotiation.cc:407] Connection header received
0926 20:40:12.784445 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:12.784455 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:12.784644 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:12.784786 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:12.792350 (+ 7564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.794126 (+ 1776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.800411 (+ 6285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.801186 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.801292 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.808792 (+ 7500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:12.808800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:12.808805 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:12.808850 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:12.816411 (+ 7561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.821258 (+ 4847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.821265 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.821268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.821434 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.822050 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.822057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.822060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.822425 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:12.822577 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:12.826032 (+ 3455us) server_negotiation.cc:299] Negotiation successful
0926 20:40:12.826299 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24848,"thread_start_us":32390,"threads_started":1}
I20260926 20:40:12.826701 31185 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:12.827219 31185 heartbeater.cc:511] Master 127.23.44.253:40781 requested a full tablet report, sending...
I20260926 20:40:12.827517 31184 heartbeater.cc:347] Connected to a master server at 127.23.44.254:41983
I20260926 20:40:12.827638 31184 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:12.828178 31184 heartbeater.cc:511] Master 127.23.44.254:41983 requested a full tablet report, sending...
I20260926 20:40:12.829203 29923 ts_manager.cc:194] Re-registered known tserver with Master: e01a9a75c042415dacc31c4cfb89a56b (127.23.44.195:33263)
I20260926 20:40:12.829340 30525 ts_manager.cc:194] Registered new tserver with Master: e01a9a75c042415dacc31c4cfb89a56b (127.23.44.195:33263)
I20260926 20:40:12.830403 29923 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:58799
I20260926 20:40:12.840519 31190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.811368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37666 (local address 127.23.44.252:34937)
0926 20:40:12.816596 (+ 5228us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:12.816602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:12.816710 (+ 108us) server_negotiation.cc:407] Connection header received
0926 20:40:12.817197 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:12.817204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:12.817325 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:12.817392 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:12.819808 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.820829 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.826810 (+ 5981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.827621 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.827754 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.832506 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:12.832513 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:12.832517 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:12.832585 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:12.836657 (+ 4072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.837607 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.837615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.837620 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.837792 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.838377 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.838386 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.838390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.838769 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:12.839075 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:12.839829 (+ 754us) server_negotiation.cc:299] Negotiation successful
0926 20:40:12.840078 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5082,"thread_start_us":19374,"threads_started":1}
I20260926 20:40:12.849944 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.804165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49664 (local address 127.23.44.254:41983)
0926 20:40:12.826921 (+ 22756us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:12.826927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:12.826943 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:40:12.826987 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:12.826992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:12.827080 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:12.827178 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:12.833081 (+ 5903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.834681 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.837621 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.838292 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.838439 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.842781 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:12.842787 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:12.842795 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:12.842842 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:12.846077 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.847008 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.847012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.847014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.847141 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.847720 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.847726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.847729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.848121 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:12.848352 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:12.848812 (+ 460us) server_negotiation.cc:299] Negotiation successful
0926 20:40:12.849117 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22653,"thread_start_us":26656,"threads_started":1}
I20260926 20:40:12.854809 31187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.829117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54728 (local address 127.23.44.253:40781)
0926 20:40:12.829334 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:12.829342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:12.838286 (+ 8944us) server_negotiation.cc:407] Connection header received
0926 20:40:12.838448 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:12.838453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:12.838542 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:12.838635 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:12.841690 (+ 3055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.842677 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.843951 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.844750 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.844855 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.848276 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:12.848283 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:12.848288 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:12.848333 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:12.851779 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.852459 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.852465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.852469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.852663 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:12.853149 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:12.853154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:12.853158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:12.853533 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:12.853802 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:12.854227 (+ 425us) server_negotiation.cc:299] Negotiation successful
0926 20:40:12.854491 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 20:40:12.854837 29923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:40:12.862486 29923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:40:12.911851 31217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.886058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:33263 (local address 127.0.0.1:48202)
0926 20:40:12.898818 (+ 12760us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:12.898839 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:12.898972 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:12.901087 (+ 2115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:12.901095 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:12.901132 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:12.901556 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.901571 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.904116 (+ 2545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.904122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.907813 (+ 3691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.907839 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.910771 (+ 2932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.910779 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.910972 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.911023 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 20:40:12.911098 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 20:40:12.911256 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12598,"threads_started":1}
I20260926 20:40:12.914084 31216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.887635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.23.44.195:33263)
0926 20:40:12.899863 (+ 12228us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:12.899877 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:12.899927 (+ 50us) server_negotiation.cc:407] Connection header received
0926 20:40:12.900073 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:12.900085 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:12.900444 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:12.900715 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:12.901842 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.903943 (+ 2101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.908116 (+ 4173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.910501 (+ 2385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.912874 (+ 2373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.913193 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:12.913285 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 20:40:12.913520 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11923,"thread_start_us":20910,"threads_started":1}
I20260926 20:40:12.919003 31118 tablet_service.cc:1467] Processing CreateTablet for tablet 0f4d842a75244af2873b8a194647c617 (DEFAULT_TABLE table=test-table [id=cae52dd15a75402db1fc749b65550a24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:12.920814 31118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f4d842a75244af2873b8a194647c617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:12.923895 31217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.886553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:39483 (local address 127.0.0.1:51214)
0926 20:40:12.912455 (+ 25902us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:12.912482 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:12.912651 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:12.915481 (+ 2830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:12.915486 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:12.915513 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:12.915717 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.915732 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.917768 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.917773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.919962 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.919983 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.922964 (+ 2981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.922971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.923221 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.923271 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 20:40:12.923370 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 20:40:12.923551 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25736,"threads_started":1}
I20260926 20:40:12.925354 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.886779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51214 (local address 127.23.44.194:39483)
0926 20:40:12.887540 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:12.887551 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:12.914884 (+ 27333us) server_negotiation.cc:407] Connection header received
0926 20:40:12.915007 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:12.915015 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:12.915289 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:12.915494 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:12.915982 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.917599 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.920212 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.922607 (+ 2395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.924362 (+ 1755us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.924663 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:12.924756 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 20:40:12.925023 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":314,"threads_started":1}
I20260926 20:40:12.933465 30977 tablet_service.cc:1467] Processing CreateTablet for tablet 0f4d842a75244af2873b8a194647c617 (DEFAULT_TABLE table=test-table [id=cae52dd15a75402db1fc749b65550a24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:12.935338 30977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f4d842a75244af2873b8a194647c617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:12.935386 31213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.888076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:35693 (local address 127.0.0.1:57472)
0926 20:40:12.914187 (+ 26111us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:12.914212 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:12.914335 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:12.924482 (+ 10147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:12.924488 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:12.924523 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:12.925034 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.925049 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.926974 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.926979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.931425 (+ 4446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:12.931449 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.934372 (+ 2923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.934380 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:12.934603 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.934645 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 20:40:12.934776 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 20:40:12.934969 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25963,"thread_start_us":11001,"threads_started":1}
I20260926 20:40:12.935700 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:12.888363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57472 (local address 127.23.44.193:35693)
0926 20:40:12.923345 (+ 34982us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:12.923357 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:12.923392 (+ 35us) server_negotiation.cc:407] Connection header received
0926 20:40:12.923513 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:12.923522 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:12.923939 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:12.925277 (+ 1338us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:12.925542 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.926824 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.931752 (+ 4928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:12.933726 (+ 1974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:12.933936 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:12.934201 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:12.934865 (+ 664us) server_negotiation.cc:299] Negotiation successful
0926 20:40:12.935132 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34724,"thread_start_us":35773,"threads_started":1}
I20260926 20:40:12.940074 30831 tablet_service.cc:1467] Processing CreateTablet for tablet 0f4d842a75244af2873b8a194647c617 (DEFAULT_TABLE table=test-table [id=cae52dd15a75402db1fc749b65550a24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:12.941768 30831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f4d842a75244af2873b8a194647c617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:12.944393 31219 tablet_bootstrap.cc:492] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b: Bootstrap starting.
I20260926 20:40:12.948297 31219 tablet_bootstrap.cc:654] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:12.949839 31219 log.cc:824] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:12.953171 31221 tablet_bootstrap.cc:492] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb: Bootstrap starting.
I20260926 20:40:12.954380 31219 tablet_bootstrap.cc:492] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b: No bootstrap required, opened a new log
I20260926 20:40:12.954718 31219 ts_tablet_manager.cc:1397] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.002s
I20260926 20:40:12.957043 31221 tablet_bootstrap.cc:654] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:12.958185 31221 log.cc:824] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:12.960860 31223 tablet_bootstrap.cc:492] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1: Bootstrap starting.
I20260926 20:40:12.961619 31221 tablet_bootstrap.cc:492] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb: No bootstrap required, opened a new log
I20260926 20:40:12.961947 31221 ts_tablet_manager.cc:1397] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260926 20:40:12.965553 31219 raft_consensus.cc:348] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:12.965960 31219 raft_consensus.cc:374] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:12.966068 31219 raft_consensus.cc:729] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01a9a75c042415dacc31c4cfb89a56b, State: Initialized, Role: FOLLOWER
I20260926 20:40:12.966171 31223 tablet_bootstrap.cc:654] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:12.966730 31219 consensus_queue.cc:260] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:12.967618 31223 log.cc:824] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:12.970280 31219 ts_tablet_manager.cc:1428] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
I20260926 20:40:12.970815 31184 heartbeater.cc:503] Master 127.23.44.254:41983 was elected leader, sending a full tablet report...
I20260926 20:40:12.971311 31223 tablet_bootstrap.cc:492] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1: No bootstrap required, opened a new log
I20260926 20:40:12.971618 31223 ts_tablet_manager.cc:1397] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260926 20:40:12.979274 31221 raft_consensus.cc:348] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:12.979782 31221 raft_consensus.cc:374] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:12.979882 31221 raft_consensus.cc:729] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a08386306b2d4fbc83729667f83d09fb, State: Initialized, Role: FOLLOWER
I20260926 20:40:12.980528 31221 consensus_queue.cc:260] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:12.982081 31043 heartbeater.cc:503] Master 127.23.44.254:41983 was elected leader, sending a full tablet report...
I20260926 20:40:12.982817 31221 ts_tablet_manager.cc:1428] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb: Time spent starting tablet: real 0.021s user 0.011s sys 0.005s
I20260926 20:40:12.984283 31223 raft_consensus.cc:348] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:12.984874 31223 raft_consensus.cc:374] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:12.984979 31223 raft_consensus.cc:729] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a32161d75a4447f7a69c09e3024251f1, State: Initialized, Role: FOLLOWER
I20260926 20:40:12.985735 31223 consensus_queue.cc:260] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:12.988889 31223 ts_tablet_manager.cc:1428] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1: Time spent starting tablet: real 0.017s user 0.017s sys 0.000s
I20260926 20:40:13.003183 31227 raft_consensus.cc:482] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:40:13.003714 31227 raft_consensus.cc:504] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:13.005896 31227 leader_election.cc:288] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01a9a75c042415dacc31c4cfb89a56b (127.23.44.195:33263), a08386306b2d4fbc83729667f83d09fb (127.23.44.194:39483)
I20260926 20:40:13.015486 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:13.006363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:44415 (local address 127.23.44.194:39483)
0926 20:40:13.006570 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:13.006577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:13.007334 (+ 757us) server_negotiation.cc:407] Connection header received
0926 20:40:13.007586 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:13.007592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:13.007674 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:13.007785 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:13.008694 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.009919 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.013303 (+ 3384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.014354 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.014521 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:13.014624 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:13.015052 (+ 428us) server_negotiation.cc:299] Negotiation successful
0926 20:40:13.015183 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:40:13.015502 31216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:13.006365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:34149 (local address 127.23.44.195:33263)
0926 20:40:13.006570 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:13.006577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:13.007335 (+ 758us) server_negotiation.cc:407] Connection header received
0926 20:40:13.007586 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:13.007592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:13.007682 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:13.007785 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:13.008694 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.009919 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.013271 (+ 3352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.014368 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.014550 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:13.014630 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:13.015118 (+ 488us) server_negotiation.cc:299] Negotiation successful
0926 20:40:13.015247 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 20:40:13.015688 31228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:13.006270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:33263 (local address 127.23.44.193:34149)
0926 20:40:13.007180 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:13.007207 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:13.007378 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:13.007817 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:13.007823 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:13.007859 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:13.008524 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:13.008534 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.010103 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.010110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:13.013059 (+ 2949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:13.013079 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.014804 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.014808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:13.014942 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:13.014965 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 20:40:13.015092 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 20:40:13.015239 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":203,"spinlock_wait_cycles":4608,"thread_start_us":487,"threads_started":1}
I20260926 20:40:13.015877 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:13.006270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:39483 (local address 127.23.44.193:44415)
0926 20:40:13.007202 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:13.007216 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:13.007378 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:13.007817 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:13.007823 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:13.007859 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:13.008460 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:13.008478 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.010103 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.010109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:13.013070 (+ 2961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:13.013079 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.014732 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.014739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:13.014870 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:13.014901 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 20:40:13.015024 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 20:40:13.015184 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"mutex_wait_us":384,"spinlock_wait_cycles":21632,"thread_start_us":483,"threads_started":1}
I20260926 20:40:13.017045 31138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f4d842a75244af2873b8a194647c617" candidate_uuid: "a32161d75a4447f7a69c09e3024251f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01a9a75c042415dacc31c4cfb89a56b" is_pre_election: true
I20260926 20:40:13.017189 30997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f4d842a75244af2873b8a194647c617" candidate_uuid: "a32161d75a4447f7a69c09e3024251f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a08386306b2d4fbc83729667f83d09fb" is_pre_election: true
I20260926 20:40:13.017621 31138 raft_consensus.cc:2454] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a32161d75a4447f7a69c09e3024251f1 in term 0.
I20260926 20:40:13.017697 30997 raft_consensus.cc:2454] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a32161d75a4447f7a69c09e3024251f1 in term 0.
I20260926 20:40:13.018379 30786 leader_election.cc:302] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a08386306b2d4fbc83729667f83d09fb, a32161d75a4447f7a69c09e3024251f1; no voters:
I20260926 20:40:13.018764 31227 raft_consensus.cc:2781] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:40:13.019366 31227 raft_consensus.cc:482] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:40:13.019477 31227 raft_consensus.cc:3037] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:13.021777 31227 raft_consensus.cc:504] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:13.022513 31227 leader_election.cc:288] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [CANDIDATE]: Term 1 election: Requested vote from peers e01a9a75c042415dacc31c4cfb89a56b (127.23.44.195:33263), a08386306b2d4fbc83729667f83d09fb (127.23.44.194:39483)
I20260926 20:40:13.022954 31138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f4d842a75244af2873b8a194647c617" candidate_uuid: "a32161d75a4447f7a69c09e3024251f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01a9a75c042415dacc31c4cfb89a56b"
I20260926 20:40:13.022966 30997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f4d842a75244af2873b8a194647c617" candidate_uuid: "a32161d75a4447f7a69c09e3024251f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a08386306b2d4fbc83729667f83d09fb"
I20260926 20:40:13.023171 31138 raft_consensus.cc:3037] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:13.023185 30997 raft_consensus.cc:3037] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:13.025507 31138 raft_consensus.cc:2454] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a32161d75a4447f7a69c09e3024251f1 in term 1.
I20260926 20:40:13.025516 30997 raft_consensus.cc:2454] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a32161d75a4447f7a69c09e3024251f1 in term 1.
I20260926 20:40:13.026016 30786 leader_election.cc:302] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a08386306b2d4fbc83729667f83d09fb, a32161d75a4447f7a69c09e3024251f1, e01a9a75c042415dacc31c4cfb89a56b; no voters:
I20260926 20:40:13.026368 31227 raft_consensus.cc:2781] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:13.026858 31227 raft_consensus.cc:686] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [term 1 LEADER]: Becoming Leader. State: Replica: a32161d75a4447f7a69c09e3024251f1, State: Running, Role: LEADER
I20260926 20:40:13.027423 31227 consensus_queue.cc:237] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:13.032861 29923 catalog_manager.cc:5495] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a32161d75a4447f7a69c09e3024251f1 (127.23.44.193). New cstate: current_term: 1 leader_uuid: "a32161d75a4447f7a69c09e3024251f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } health_report { overall_health: HEALTHY } } }
W20260926 20:40:13.046809 31186 tablet.cc:2369] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:40:13.062062 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:13.055195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57480 (local address 127.23.44.193:35693)
0926 20:40:13.055811 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:13.055817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:13.055843 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:40:13.055980 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:13.055984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:13.056071 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:13.056232 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:40:13.056753 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.057865 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.059475 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.060006 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.060169 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:13.060815 (+ 646us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:40:13.061369 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:40:13.061501 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:13.061629 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 20:40:13.061721 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433}
I20260926 20:40:13.075335 31138 raft_consensus.cc:1261] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b [term 1 FOLLOWER]: Refusing update from remote peer a32161d75a4447f7a69c09e3024251f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:13.077106 30997 raft_consensus.cc:1261] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb [term 1 FOLLOWER]: Refusing update from remote peer a32161d75a4447f7a69c09e3024251f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:13.077096 31227 consensus_queue.cc:1035] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:13.078307 31227 consensus_queue.cc:1035] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:13.098897 31233 mvcc.cc:205] Tried to move back new op lower bound from 7333704552734396416 to 7333704552564436992. Current Snapshot: MvccSnapshot[applied={T|T < 7333704552734396416}]
I20260926 20:40:13.101011 31235 mvcc.cc:205] Tried to move back new op lower bound from 7333704552734396416 to 7333704552564436992. Current Snapshot: MvccSnapshot[applied={T|T < 7333704552734396416}]
I20260926 20:40:13.102075 31234 mvcc.cc:205] Tried to move back new op lower bound from 7333704552734396416 to 7333704552564436992. Current Snapshot: MvccSnapshot[applied={T|T < 7333704552734396416}]
I20260926 20:40:13.130267 29923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:13.130606 29923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:13.139770 29923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c12f5c6e8bf842719909b3f57df1bbf0: Sending DeleteTablet for 3 replicas of tablet 0f4d842a75244af2873b8a194647c617
I20260926 20:40:13.141080 31118 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f4d842a75244af2873b8a194647c617 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:13 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:48202
I20260926 20:40:13.144091 31243 tablet_replica.cc:330] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b: stopping tablet replica
I20260926 20:40:13.144650 31243 raft_consensus.cc:2229] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:13.145248 30977 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f4d842a75244af2873b8a194647c617 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:13 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:51214
I20260926 20:40:13.145364 31243 raft_consensus.cc:2258] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:13.146554 31245 tablet_replica.cc:330] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb: stopping tablet replica
I20260926 20:40:13.146986 31245 raft_consensus.cc:2229] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:13.147522 30831 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f4d842a75244af2873b8a194647c617 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:13 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:57472
I20260926 20:40:13.147626 31245 raft_consensus.cc:2258] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:13.148444 31246 tablet_replica.cc:330] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1: stopping tablet replica
I20260926 20:40:13.148927 31246 raft_consensus.cc:2229] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:40:13.149596 31246 raft_consensus.cc:2258] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:13.149741 31245 ts_tablet_manager.cc:1905] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:13.151312 31243 ts_tablet_manager.cc:1905] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:13.153121 31246 ts_tablet_manager.cc:1905] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:13.158946 31245 ts_tablet_manager.cc:1918] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:13.159164 31245 log.cc:1196] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/0f4d842a75244af2873b8a194647c617
I20260926 20:40:13.159811 31245 ts_tablet_manager.cc:1939] T 0f4d842a75244af2873b8a194647c617 P a08386306b2d4fbc83729667f83d09fb: Deleting consensus metadata
I20260926 20:40:13.162257 29909 catalog_manager.cc:4841] TS a08386306b2d4fbc83729667f83d09fb (127.23.44.194:39483): tablet 0f4d842a75244af2873b8a194647c617 (table test-table [id=cae52dd15a75402db1fc749b65550a24]) successfully deleted
I20260926 20:40:13.166666 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 29891
I20260926 20:40:13.167909 29953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:13.167896 31246 ts_tablet_manager.cc:1918] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:13.168231 31246 log.cc:1196] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/0f4d842a75244af2873b8a194647c617
I20260926 20:40:13.169102 31246 ts_tablet_manager.cc:1939] T 0f4d842a75244af2873b8a194647c617 P a32161d75a4447f7a69c09e3024251f1: Deleting consensus metadata
I20260926 20:40:13.167058 31243 ts_tablet_manager.cc:1918] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:13.173887 31243 log.cc:1196] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/0f4d842a75244af2873b8a194647c617
I20260926 20:40:13.174593 31243 ts_tablet_manager.cc:1939] T 0f4d842a75244af2873b8a194647c617 P e01a9a75c042415dacc31c4cfb89a56b: Deleting consensus metadata
I20260926 20:40:13.193310 31187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:13.144336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54740 (local address 127.23.44.253:40781)
0926 20:40:13.144521 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:13.144528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:13.164475 (+ 19947us) server_negotiation.cc:407] Connection header received
0926 20:40:13.164800 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:13.164806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:13.164901 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:13.165008 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:13.168900 (+ 3892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.170873 (+ 1973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.173985 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.174730 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.174869 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:13.184052 (+ 9183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:13.184062 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:13.184069 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:13.184121 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:13.189288 (+ 5167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:13.190725 (+ 1437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:13.190731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:13.190735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:13.190910 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:13.191389 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:13.191395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:13.191400 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:13.191852 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:13.192136 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:13.192626 (+ 490us) server_negotiation.cc:299] Negotiation successful
0926 20:40:13.192898 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 20:40:13.213153 31190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:13.165402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37682 (local address 127.23.44.252:34937)
0926 20:40:13.165612 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:13.165618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:13.193416 (+ 27798us) server_negotiation.cc:407] Connection header received
0926 20:40:13.194603 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:13.194609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:13.194718 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:13.194847 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:13.196786 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.198447 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.200506 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.201215 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.201346 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:13.205149 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:13.205156 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:13.205162 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:13.205214 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:13.209297 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:13.210394 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:13.210400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:13.210404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:13.210572 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:13.211026 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:13.211031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:13.211035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:13.211401 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:13.211674 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:13.212043 (+ 369us) server_negotiation.cc:299] Negotiation successful
0926 20:40:13.212270 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 20:40:13.308898 29910 catalog_manager.cc:4841] TS e01a9a75c042415dacc31c4cfb89a56b (127.23.44.195:33263): tablet 0f4d842a75244af2873b8a194647c617 (table test-table [id=cae52dd15a75402db1fc749b65550a24]) successfully deleted
I20260926 20:40:13.308815 29907 catalog_manager.cc:4841] TS a32161d75a4447f7a69c09e3024251f1 (127.23.44.193:35693): tablet 0f4d842a75244af2873b8a194647c617 (table test-table [id=cae52dd15a75402db1fc749b65550a24]) successfully deleted
W20260926 20:40:13.309269 29910 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.23.44.195:33263, user_credentials={real_user=slave}} blocked reactor thread for 132620us
W20260926 20:40:13.309371 29907 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.23.44.193:35693, user_credentials={real_user=slave}} blocked reactor thread for 138174us
I20260926 20:40:13.310775 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 29891
I20260926 20:40:13.371901 31187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:13.343178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54750 (local address 127.23.44.253:40781)
0926 20:40:13.343447 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:13.343456 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:13.354639 (+ 11183us) server_negotiation.cc:407] Connection header received
0926 20:40:13.354817 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:13.354826 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:13.354931 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:13.355041 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:13.357523 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.358790 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.360431 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.361140 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.361266 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:13.364696 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:13.364703 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:13.364707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:13.364746 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:13.367732 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:13.368430 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:13.368436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:13.368440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:13.368642 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:13.369127 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:13.369134 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:13.369137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:13.369546 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:13.369778 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:13.371280 (+ 1502us) server_negotiation.cc:299] Negotiation successful
0926 20:40:13.371524 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260926 20:40:13.373229 31190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:13.342549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37684 (local address 127.23.44.252:34937)
0926 20:40:13.342839 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:13.342849 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:13.351436 (+ 8587us) server_negotiation.cc:407] Connection header received
0926 20:40:13.352090 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:13.352101 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:13.352215 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:13.352310 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:13.356229 (+ 3919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.357397 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.359396 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:13.360016 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:13.360157 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:13.364157 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:13.364163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:13.364168 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:13.364206 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:13.368542 (+ 4336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:13.369458 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:13.369463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:13.369467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:13.369614 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:13.370752 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:13.370758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:13.370761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:13.371174 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:13.371438 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:13.372581 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0926 20:40:13.372828 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 20:40:14.113379 31255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.112531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.193:40933)
0926 20:40:14.113061 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.113186 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":254,"threads_started":1}
W20260926 20:40:14.113782 30897 heartbeater.cc:644] Failed to heartbeat to 127.23.44.254:41983 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
I20260926 20:40:14.121608 31256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.119444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.194:55809)
0926 20:40:14.121264 (+ 1820us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.121400 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1710,"thread_start_us":595,"threads_started":1}
W20260926 20:40:14.122072 31043 heartbeater.cc:644] Failed to heartbeat to 127.23.44.254:41983 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
I20260926 20:40:14.125249 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.124415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.195:45433)
0926 20:40:14.124868 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.124988 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":255,"threads_started":1}
W20260926 20:40:14.125670 31184 heartbeater.cc:644] Failed to heartbeat to 127.23.44.254:41983 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
I20260926 20:40:14.640365 31258 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c12f5c6e8bf842719909b3f57df1bbf0)
I20260926 20:40:14.640728 31258 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } }
I20260926 20:40:14.640851 31259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c12f5c6e8bf842719909b3f57df1bbf0)
I20260926 20:40:14.641244 31259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } }
I20260926 20:40:14.642369 31258 leader_election.cc:288] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c12f5c6e8bf842719909b3f57df1bbf0 (127.23.44.254:41983), 608b462fd1104559963abb6596143116 (127.23.44.252:34937)
I20260926 20:40:14.643175 31259 leader_election.cc:288] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c12f5c6e8bf842719909b3f57df1bbf0 (127.23.44.254:41983), a6a97c669fd2448c8c74d85265130e1e (127.23.44.253:40781)
I20260926 20:40:14.644438 31261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.642506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:50042)
0926 20:40:14.643855 (+ 1349us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.644122 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1186,"threads_started":1}
W20260926 20:40:14.652459 30509 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
I20260926 20:40:14.654911 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.645277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:50050)
0926 20:40:14.654065 (+ 8788us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.654472 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8552,"thread_start_us":8256,"threads_started":1}
W20260926 20:40:14.655494 30700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
I20260926 20:40:14.665046 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.664604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:50076)
0926 20:40:14.664767 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.664869 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 20:40:14.666025 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.643498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:40781 (local address 127.0.0.1:54760)
0926 20:40:14.645935 (+ 2437us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.645958 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:14.646099 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:14.656320 (+ 10221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:14.656325 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:14.656385 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:14.656779 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.656794 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.659218 (+ 2424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.659227 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.662431 (+ 3204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.662453 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.665268 (+ 2815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.665274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.665462 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.665500 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 20:40:14.665572 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 20:40:14.665715 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2273,"thread_start_us":2109,"threads_started":1}
I20260926 20:40:14.666386 31260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.665777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:50082)
0926 20:40:14.665938 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.666133 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260926 20:40:14.666522 30700 leader_election.cc:334] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c12f5c6e8bf842719909b3f57df1bbf0 (127.23.44.254:41983): Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
I20260926 20:40:14.667025 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.643942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54760 (local address 127.23.44.253:40781)
0926 20:40:14.655859 (+ 11917us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:14.655866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:14.655889 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:40:14.655966 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:14.655974 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:14.656050 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:14.656979 (+ 929us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:14.657305 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.658958 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.662702 (+ 3744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.664316 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.666387 (+ 2071us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.666493 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:14.666547 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 20:40:14.666689 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11795,"thread_start_us":12175,"threads_started":1}
W20260926 20:40:14.667089 30509 leader_election.cc:334] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c12f5c6e8bf842719909b3f57df1bbf0 (127.23.44.254:41983): Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
I20260926 20:40:14.667551 30535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "608b462fd1104559963abb6596143116" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a6a97c669fd2448c8c74d85265130e1e" is_pre_election: true
I20260926 20:40:14.667919 30535 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 608b462fd1104559963abb6596143116 in term 1.
I20260926 20:40:14.668720 30703 leader_election.cc:302] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 608b462fd1104559963abb6596143116, a6a97c669fd2448c8c74d85265130e1e; no voters: c12f5c6e8bf842719909b3f57df1bbf0
I20260926 20:40:14.668865 31261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.642657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:34937 (local address 127.0.0.1:37694)
0926 20:40:14.644697 (+ 2040us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.644720 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:14.644894 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:14.660789 (+ 15895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:14.660797 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:14.660950 (+ 153us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:14.661352 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.661369 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.664051 (+ 2682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.664059 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.666385 (+ 2326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.666400 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.667999 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.668003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.668270 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.668295 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 20:40:14.668406 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 20:40:14.668549 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1920,"thread_start_us":891,"threads_started":1}
I20260926 20:40:14.669064 31262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.643040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37694 (local address 127.23.44.252:34937)
0926 20:40:14.659649 (+ 16609us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:14.659657 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:14.659683 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:40:14.659768 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:14.659775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:14.660231 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:14.661591 (+ 1360us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:14.662002 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.663825 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.666606 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.667802 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.667945 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.668045 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:14.668429 (+ 384us) server_negotiation.cc:299] Negotiation successful
0926 20:40:14.668585 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16401,"thread_start_us":17446,"threads_started":1}
I20260926 20:40:14.669209 31259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 20:40:14.669334 31259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c12f5c6e8bf842719909b3f57df1bbf0)
I20260926 20:40:14.669471 31259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 1 FOLLOWER]: Advancing to term 2
I20260926 20:40:14.669742 30726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a6a97c669fd2448c8c74d85265130e1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "608b462fd1104559963abb6596143116" is_pre_election: true
I20260926 20:40:14.671700 31259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } }
I20260926 20:40:14.672173 30726 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6a97c669fd2448c8c74d85265130e1e in current term 2: Already voted for candidate 608b462fd1104559963abb6596143116 in this term.
I20260926 20:40:14.672418 31259 leader_election.cc:288] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [CANDIDATE]: Term 2 election: Requested vote from peers c12f5c6e8bf842719909b3f57df1bbf0 (127.23.44.254:41983), a6a97c669fd2448c8c74d85265130e1e (127.23.44.253:40781)
I20260926 20:40:14.672912 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.672513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:50096)
0926 20:40:14.672668 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.672765 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 20:40:14.673269 30535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "608b462fd1104559963abb6596143116" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a6a97c669fd2448c8c74d85265130e1e"
I20260926 20:40:14.673522 30535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 1 FOLLOWER]: Advancing to term 2
I20260926 20:40:14.673484 30510 leader_election.cc:302] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6a97c669fd2448c8c74d85265130e1e; no voters: 608b462fd1104559963abb6596143116, c12f5c6e8bf842719909b3f57df1bbf0
I20260926 20:40:14.673897 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.673508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:50112)
0926 20:40:14.673653 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.673768 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260926 20:40:14.674081 30700 leader_election.cc:334] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c12f5c6e8bf842719909b3f57df1bbf0 (127.23.44.254:41983): Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
I20260926 20:40:14.675850 30535 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 608b462fd1104559963abb6596143116 in term 2.
I20260926 20:40:14.676087 31258 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 20:40:14.676285 30703 leader_election.cc:302] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 608b462fd1104559963abb6596143116, a6a97c669fd2448c8c74d85265130e1e; no voters: c12f5c6e8bf842719909b3f57df1bbf0
I20260926 20:40:14.676616 31259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 20:40:14.677073 31259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [term 2 LEADER]: Becoming Leader. State: Replica: 608b462fd1104559963abb6596143116, State: Running, Role: LEADER
I20260926 20:40:14.677441 31259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } }
I20260926 20:40:14.679718 31268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 608b462fd1104559963abb6596143116. Latest consensus state: current_term: 2 leader_uuid: "608b462fd1104559963abb6596143116" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } } }
I20260926 20:40:14.680127 31268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:14.680943 31270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:40:14.684900 31270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:40:14.685819 31270 catalog_manager.cc:1259] Loaded cluster ID: 7d81cd72aff04fd7b14a2d74ebb5d49b
I20260926 20:40:14.685894 31270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:40:14.687106 31270 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:40:14.687925 31270 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: Loaded TSK: 0
I20260926 20:40:14.688887 31270 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:40:14.724375 30716 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:40:14.731441 30716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:40:14.736126 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.735628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:50128)
0926 20:40:14.735793 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.735943 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 20:40:14.736097 30535 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [term 2 FOLLOWER]: Refusing update from remote peer 608b462fd1104559963abb6596143116: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260926 20:40:14.737078 31259 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260926 20:40:14.737447 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.737005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:50140)
0926 20:40:14.737190 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.737293 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 20:40:14.737658 30700 consensus_peers.cc:487] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 -> Peer c12f5c6e8bf842719909b3f57df1bbf0 (127.23.44.254:41983): Couldn't send request to peer c12f5c6e8bf842719909b3f57df1bbf0. Status: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 20:40:14.740697 31258 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 608b462fd1104559963abb6596143116. Latest consensus state: current_term: 2 leader_uuid: "608b462fd1104559963abb6596143116" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } } }
I20260926 20:40:14.741037 31258 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:40:14.743360 31259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "608b462fd1104559963abb6596143116" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } } }
I20260926 20:40:14.743734 31259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:14.744261 31258 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "608b462fd1104559963abb6596143116" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c12f5c6e8bf842719909b3f57df1bbf0" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41983 } } peers { permanent_uuid: "a6a97c669fd2448c8c74d85265130e1e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 40781 } attrs { promote: false } } peers { permanent_uuid: "608b462fd1104559963abb6596143116" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 34937 } attrs { promote: false } } }
I20260926 20:40:14.744675 31258 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a97c669fd2448c8c74d85265130e1e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:40:14.763441 31277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.754421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.23.44.194:39483)
0926 20:40:14.755403 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:14.755409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:14.755426 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:40:14.755500 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:14.755505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:14.755595 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:14.756270 (+ 675us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:14.756469 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.757949 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.761446 (+ 3497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.762415 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.762572 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.762687 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:14.763022 (+ 335us) server_negotiation.cc:299] Negotiation successful
0926 20:40:14.763170 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":868,"thread_start_us":1001,"threads_started":1}
I20260926 20:40:14.763497 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.754304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:39483 (local address 127.0.0.1:51218)
0926 20:40:14.754457 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.754477 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:14.754581 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:14.755792 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:14.755800 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:14.755841 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:14.756107 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.756124 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.758162 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.758171 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.760918 (+ 2747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.760944 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.762584 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.762589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.762842 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.762879 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 20:40:14.762991 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 20:40:14.763148 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":5}
I20260926 20:40:14.764433 30977 tablet_service.cc:1467] Processing CreateTablet for tablet 8eba692ee12d4bd58937d83c8d43aaa1 (DEFAULT_TABLE table=test-table [id=447517ec61524d16be0ef981ed182fe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:14.764918 30977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8eba692ee12d4bd58937d83c8d43aaa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:14.767002 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.754463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:33263 (local address 127.0.0.1:48206)
0926 20:40:14.754627 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.754650 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:14.754775 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:14.755643 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:14.755651 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:14.755687 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:14.756134 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.756149 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.758163 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.758171 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.763648 (+ 5477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.763661 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.766223 (+ 2562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.766229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.766405 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.766445 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 20:40:14.766540 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 20:40:14.766693 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":2925}
I20260926 20:40:14.767060 31278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.754781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.23.44.195:33263)
0926 20:40:14.755239 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:14.755245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:14.755264 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:40:14.755344 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:14.755349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:14.755434 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:14.756301 (+ 867us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:14.756516 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.757968 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.763989 (+ 6021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.765183 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.765313 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.765441 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:14.766620 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0926 20:40:14.766775 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":212,"threads_started":1}
I20260926 20:40:14.768142 31118 tablet_service.cc:1467] Processing CreateTablet for tablet 8eba692ee12d4bd58937d83c8d43aaa1 (DEFAULT_TABLE table=test-table [id=447517ec61524d16be0ef981ed182fe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:14.768635 31118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8eba692ee12d4bd58937d83c8d43aaa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:14.769712 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.755320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:35693 (local address 127.0.0.1:57488)
0926 20:40:14.761095 (+ 5775us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.761114 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:14.761234 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:14.763352 (+ 2118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:14.763356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:14.763379 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:14.763878 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.763890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.765198 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.765203 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.767089 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.767104 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.768762 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.768769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.768945 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.768982 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 20:40:14.769089 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 20:40:14.769323 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5651,"thread_start_us":5498,"threads_started":1}
I20260926 20:40:14.770469 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.755424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57488 (local address 127.23.44.193:35693)
0926 20:40:14.763044 (+ 7620us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:14.763050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:14.763068 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:40:14.763135 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:14.763140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:14.763209 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:14.764034 (+ 825us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:14.764175 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.765045 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.767299 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.768478 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.768661 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.768768 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:14.770025 (+ 1257us) server_negotiation.cc:299] Negotiation successful
0926 20:40:14.770197 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7498,"thread_start_us":7393,"threads_started":1}
I20260926 20:40:14.771339 30831 tablet_service.cc:1467] Processing CreateTablet for tablet 8eba692ee12d4bd58937d83c8d43aaa1 (DEFAULT_TABLE table=test-table [id=447517ec61524d16be0ef981ed182fe7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:14.771559 31281 tablet_bootstrap.cc:492] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb: Bootstrap starting.
I20260926 20:40:14.771775 30831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8eba692ee12d4bd58937d83c8d43aaa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:14.773372 31281 tablet_bootstrap.cc:654] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:14.775722 31283 tablet_bootstrap.cc:492] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b: Bootstrap starting.
I20260926 20:40:14.776247 31281 tablet_bootstrap.cc:492] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb: No bootstrap required, opened a new log
I20260926 20:40:14.776388 31281 ts_tablet_manager.cc:1397] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 20:40:14.777171 31281 raft_consensus.cc:348] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:14.777390 31283 tablet_bootstrap.cc:654] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:14.777462 31281 raft_consensus.cc:374] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:14.777601 31281 raft_consensus.cc:729] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a08386306b2d4fbc83729667f83d09fb, State: Initialized, Role: FOLLOWER
I20260926 20:40:14.777920 31281 consensus_queue.cc:260] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:14.785496 31284 tablet_bootstrap.cc:492] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1: Bootstrap starting.
I20260926 20:40:14.786120 31281 ts_tablet_manager.cc:1428] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb: Time spent starting tablet: real 0.010s user 0.000s sys 0.010s
I20260926 20:40:14.787426 31283 tablet_bootstrap.cc:492] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b: No bootstrap required, opened a new log
I20260926 20:40:14.787593 31283 ts_tablet_manager.cc:1397] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.011s
I20260926 20:40:14.788506 31283 raft_consensus.cc:348] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:14.788787 31283 raft_consensus.cc:374] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:14.788837 31284 tablet_bootstrap.cc:654] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:14.788884 31283 raft_consensus.cc:729] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01a9a75c042415dacc31c4cfb89a56b, State: Initialized, Role: FOLLOWER
I20260926 20:40:14.789119 31283 consensus_queue.cc:260] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:14.790580 31042 heartbeater.cc:503] Master 127.23.44.252:34937 was elected leader, sending a full tablet report...
I20260926 20:40:14.797720 31287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.787997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.194:42629)
0926 20:40:14.797239 (+ 9242us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.797425 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9046,"thread_start_us":8843,"threads_started":1}
I20260926 20:40:14.797704 31283 ts_tablet_manager.cc:1428] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260926 20:40:14.799067 31284 tablet_bootstrap.cc:492] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1: No bootstrap required, opened a new log
I20260926 20:40:14.799454 31284 ts_tablet_manager.cc:1397] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260926 20:40:14.800294 31284 raft_consensus.cc:348] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:14.800536 31284 raft_consensus.cc:374] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:14.800679 31284 raft_consensus.cc:729] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a32161d75a4447f7a69c09e3024251f1, State: Initialized, Role: FOLLOWER
I20260926 20:40:14.801020 31183 heartbeater.cc:503] Master 127.23.44.252:34937 was elected leader, sending a full tablet report...
I20260926 20:40:14.800894 31284 consensus_queue.cc:260] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:14.802651 31290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.798613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.195:45177)
0926 20:40:14.802099 (+ 3486us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.802292 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3289,"thread_start_us":3158,"threads_started":1}
I20260926 20:40:14.803622 31290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.803237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.195:34585)
0926 20:40:14.803405 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.803487 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 20:40:14.803704 31284 ts_tablet_manager.cc:1428] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260926 20:40:14.805529 30896 heartbeater.cc:503] Master 127.23.44.252:34937 was elected leader, sending a full tablet report...
I20260926 20:40:14.805877 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.804047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.193:46199)
0926 20:40:14.805249 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.805496 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1039,"thread_start_us":889,"threads_started":1}
I20260926 20:40:14.807199 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.806672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.193:52151)
0926 20:40:14.806873 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.806993 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260926 20:40:14.860268 31045 tablet.cc:2369] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:40:14.894656 31288 raft_consensus.cc:482] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:40:14.894882 31288 raft_consensus.cc:504] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:14.896409 31288 leader_election.cc:288] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a08386306b2d4fbc83729667f83d09fb (127.23.44.194:39483), a32161d75a4447f7a69c09e3024251f1 (127.23.44.193:35693)
I20260926 20:40:14.903914 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.897052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:34101 (local address 127.23.44.193:35693)
0926 20:40:14.897364 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:14.897376 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:14.897838 (+ 462us) server_negotiation.cc:407] Connection header received
0926 20:40:14.898030 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:14.898035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:14.898099 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:14.898178 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:14.898639 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.899890 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.901741 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.902837 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.903022 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.903105 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:14.903501 (+ 396us) server_negotiation.cc:299] Negotiation successful
0926 20:40:14.903647 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260926 20:40:14.904225 31277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.896911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:34417 (local address 127.23.44.194:39483)
0926 20:40:14.897202 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:14.897211 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:14.897237 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:40:14.897568 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:14.897574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:14.897679 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:14.897777 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:14.898300 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.899626 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.902134 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.903308 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.903488 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.903566 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:14.903828 (+ 262us) server_negotiation.cc:299] Negotiation successful
0926 20:40:14.903959 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260926 20:40:14.904263 31290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.896671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:39483 (local address 127.23.44.195:34417)
0926 20:40:14.896896 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.897150 (+ 254us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:14.897362 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:14.897837 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:14.897843 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:14.897864 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:14.898113 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.898122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.899805 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.899810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.901939 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.901954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.903490 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.903495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.903675 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.903697 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 20:40:14.903803 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 20:40:14.903951 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260926 20:40:14.904467 31286 raft_consensus.cc:482] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:40:14.904702 31286 raft_consensus.cc:504] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:14.904968 30997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8eba692ee12d4bd58937d83c8d43aaa1" candidate_uuid: "e01a9a75c042415dacc31c4cfb89a56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a08386306b2d4fbc83729667f83d09fb" is_pre_election: true
I20260926 20:40:14.905290 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.896965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:35693 (local address 127.23.44.195:34101)
0926 20:40:14.897698 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.897713 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:14.897856 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:14.898230 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:14.898234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:14.898260 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:14.898501 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.898509 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.900053 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.900060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.901548 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.901562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.903037 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.903043 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.903303 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.903339 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 20:40:14.904115 (+ 776us) client_negotiation.cc:240] Negotiation successful
0926 20:40:14.904950 (+ 835us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":1,"thread_start_us":468,"threads_started":1}
I20260926 20:40:14.906136 30851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8eba692ee12d4bd58937d83c8d43aaa1" candidate_uuid: "e01a9a75c042415dacc31c4cfb89a56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a32161d75a4447f7a69c09e3024251f1" is_pre_election: true
I20260926 20:40:14.906517 30997 raft_consensus.cc:2454] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01a9a75c042415dacc31c4cfb89a56b in term 0.
I20260926 20:40:14.906545 30851 raft_consensus.cc:2454] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01a9a75c042415dacc31c4cfb89a56b in term 0.
I20260926 20:40:14.906695 31286 leader_election.cc:288] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e01a9a75c042415dacc31c4cfb89a56b (127.23.44.195:33263), a32161d75a4447f7a69c09e3024251f1 (127.23.44.193:35693)
I20260926 20:40:14.907191 31071 leader_election.cc:302] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a32161d75a4447f7a69c09e3024251f1, e01a9a75c042415dacc31c4cfb89a56b; no voters:
I20260926 20:40:14.907577 31288 raft_consensus.cc:2781] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:40:14.907691 31288 raft_consensus.cc:482] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:40:14.907779 31288 raft_consensus.cc:3037] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:14.912899 31288 raft_consensus.cc:504] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:14.914151 30997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8eba692ee12d4bd58937d83c8d43aaa1" candidate_uuid: "e01a9a75c042415dacc31c4cfb89a56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a08386306b2d4fbc83729667f83d09fb"
I20260926 20:40:14.914362 30997 raft_consensus.cc:3037] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:14.915426 31278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.907808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:58965 (local address 127.23.44.195:33263)
0926 20:40:14.907969 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:14.907975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:14.907996 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:40:14.908069 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:14.908076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:14.908174 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:14.908255 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:14.909113 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.910415 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.913053 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.914289 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.914408 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.914488 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:14.915055 (+ 567us) server_negotiation.cc:299] Negotiation successful
0926 20:40:14.915191 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 20:40:14.915750 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.908460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:40625 (local address 127.23.44.193:35693)
0926 20:40:14.908667 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:14.908672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:14.908689 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:40:14.908787 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:14.908792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:14.908863 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:14.908948 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:14.909596 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.910798 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.913291 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.914518 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.914656 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.914733 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:14.915343 (+ 610us) server_negotiation.cc:299] Negotiation successful
0926 20:40:14.915479 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 20:40:14.916106 31287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.906853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:33263 (local address 127.23.44.194:58965)
0926 20:40:14.907184 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.907203 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:14.907315 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:14.908691 (+ 1376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:14.908696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:14.908722 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:14.908963 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.908972 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.910579 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.910584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.912880 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.912894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.914663 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.914670 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.914847 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.914884 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 20:40:14.915619 (+ 735us) client_negotiation.cc:240] Negotiation successful
0926 20:40:14.915790 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":67}
I20260926 20:40:14.916510 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.907367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:35693 (local address 127.23.44.194:40625)
0926 20:40:14.908014 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.908031 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:14.908164 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:14.909110 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:14.909114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:14.909143 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:14.909373 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.909380 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.910959 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.910964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.913114 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:14.913140 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.914938 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.914945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:14.915149 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.915179 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 20:40:14.915960 (+ 781us) client_negotiation.cc:240] Negotiation successful
0926 20:40:14.916103 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":4,"thread_start_us":1102,"threads_started":1}
I20260926 20:40:14.916687 31138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8eba692ee12d4bd58937d83c8d43aaa1" candidate_uuid: "a08386306b2d4fbc83729667f83d09fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e01a9a75c042415dacc31c4cfb89a56b" is_pre_election: true
I20260926 20:40:14.916951 31138 raft_consensus.cc:2379] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a08386306b2d4fbc83729667f83d09fb in current term 1: Already voted for candidate e01a9a75c042415dacc31c4cfb89a56b in this term.
I20260926 20:40:14.917356 30997 raft_consensus.cc:2454] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01a9a75c042415dacc31c4cfb89a56b in term 1.
I20260926 20:40:14.917615 30851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8eba692ee12d4bd58937d83c8d43aaa1" candidate_uuid: "a08386306b2d4fbc83729667f83d09fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a32161d75a4447f7a69c09e3024251f1" is_pre_election: true
I20260926 20:40:14.917824 30851 raft_consensus.cc:2454] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a08386306b2d4fbc83729667f83d09fb in term 0.
I20260926 20:40:14.917893 31073 leader_election.cc:302] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a08386306b2d4fbc83729667f83d09fb, e01a9a75c042415dacc31c4cfb89a56b; no voters:
I20260926 20:40:14.918257 30930 leader_election.cc:302] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a08386306b2d4fbc83729667f83d09fb, a32161d75a4447f7a69c09e3024251f1; no voters: e01a9a75c042415dacc31c4cfb89a56b
I20260926 20:40:14.918905 31286 raft_consensus.cc:2741] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260926 20:40:14.919152 31288 leader_election.cc:288] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [CANDIDATE]: Term 1 election: Requested vote from peers a08386306b2d4fbc83729667f83d09fb (127.23.44.194:39483), a32161d75a4447f7a69c09e3024251f1 (127.23.44.193:35693)
I20260926 20:40:14.919217 30851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8eba692ee12d4bd58937d83c8d43aaa1" candidate_uuid: "e01a9a75c042415dacc31c4cfb89a56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a32161d75a4447f7a69c09e3024251f1"
I20260926 20:40:14.919389 30851 raft_consensus.cc:3037] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:14.919577 31288 raft_consensus.cc:2781] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:14.919801 31288 raft_consensus.cc:686] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 1 LEADER]: Becoming Leader. State: Replica: e01a9a75c042415dacc31c4cfb89a56b, State: Running, Role: LEADER
I20260926 20:40:14.920342 31288 consensus_queue.cc:237] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } } peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } }
I20260926 20:40:14.922391 30851 raft_consensus.cc:2454] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01a9a75c042415dacc31c4cfb89a56b in term 1.
I20260926 20:40:14.925709 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.925118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.195:60771)
0926 20:40:14.925369 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.925498 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260926 20:40:14.926728 30715 catalog_manager.cc:5495] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b reported cstate change: term changed from 0 to 1, leader changed from <none> to e01a9a75c042415dacc31c4cfb89a56b (127.23.44.195). New cstate: current_term: 1 leader_uuid: "e01a9a75c042415dacc31c4cfb89a56b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01a9a75c042415dacc31c4cfb89a56b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 } health_report { overall_health: UNKNOWN } } }
W20260926 20:40:14.930385 30899 tablet.cc:2369] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:40:14.975602 31278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.969002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48208 (local address 127.23.44.195:33263)
0926 20:40:14.969164 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:14.969169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:14.969186 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:40:14.969490 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:14.969495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:14.969617 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:14.969863 (+ 246us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:40:14.970182 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.970958 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.972548 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:14.973105 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:14.973226 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:14.973819 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:40:14.974457 (+ 638us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:40:14.974593 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:14.974820 (+ 227us) server_negotiation.cc:299] Negotiation successful
0926 20:40:14.975006 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 20:40:14.980293 30997 raft_consensus.cc:1261] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [term 1 FOLLOWER]: Refusing update from remote peer e01a9a75c042415dacc31c4cfb89a56b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:14.980682 30851 raft_consensus.cc:1261] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1 [term 1 FOLLOWER]: Refusing update from remote peer e01a9a75c042415dacc31c4cfb89a56b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:14.981101 31295 consensus_queue.cc:1035] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a08386306b2d4fbc83729667f83d09fb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 39483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:14.981501 31288 consensus_queue.cc:1035] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a32161d75a4447f7a69c09e3024251f1" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:14.984117 31287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.983431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.194:39627)
0926 20:40:14.983688 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.983842 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260926 20:40:14.986784 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.984591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.193:40639)
0926 20:40:14.985112 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.985258 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344}
I20260926 20:40:14.991227 31287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.989852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.194:59599)
0926 20:40:14.990940 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.991058 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":959}
I20260926 20:40:14.992071 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:14.991593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.195:34861)
0926 20:40:14.991783 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:14.991883 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260926 20:40:15.005007 30715 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:15.005329 30715 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:15.006533 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:15.005957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.195:45089)
0926 20:40:15.006209 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:15.006342 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260926 20:40:15.009833 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:15.008399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.23.44.193:56481)
0926 20:40:15.009461 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:15.009590 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":894}
I20260926 20:40:15.016217 30715 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 608b462fd1104559963abb6596143116: Sending DeleteTablet for 3 replicas of tablet 8eba692ee12d4bd58937d83c8d43aaa1
I20260926 20:40:15.017716 30977 tablet_service.cc:1514] Processing DeleteTablet for tablet 8eba692ee12d4bd58937d83c8d43aaa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:15 UTC) from {username='oryx', principal='oryx/127.23.44.252@KRBTEST.COM'} at 127.0.0.1:51218
I20260926 20:40:15.017823 31118 tablet_service.cc:1514] Processing DeleteTablet for tablet 8eba692ee12d4bd58937d83c8d43aaa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:15 UTC) from {username='oryx', principal='oryx/127.23.44.252@KRBTEST.COM'} at 127.0.0.1:48206
I20260926 20:40:15.018330 30831 tablet_service.cc:1514] Processing DeleteTablet for tablet 8eba692ee12d4bd58937d83c8d43aaa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:15 UTC) from {username='oryx', principal='oryx/127.23.44.252@KRBTEST.COM'} at 127.0.0.1:57488
I20260926 20:40:15.018230 31117 tablet_service.cc:1514] Processing DeleteTablet for tablet 8eba692ee12d4bd58937d83c8d43aaa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:15 UTC) from {username='oryx', principal='oryx/127.23.44.252@KRBTEST.COM'} at 127.0.0.1:48206
I20260926 20:40:15.018798 30830 tablet_service.cc:1514] Processing DeleteTablet for tablet 8eba692ee12d4bd58937d83c8d43aaa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:15 UTC) from {username='oryx', principal='oryx/127.23.44.252@KRBTEST.COM'} at 127.0.0.1:57488
I20260926 20:40:15.022930 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 30746
I20260926 20:40:15.029206 31308 tablet_replica.cc:330] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b: stopping tablet replica
I20260926 20:40:15.029498 31308 raft_consensus.cc:2229] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:40:15.030320 31308 raft_consensus.cc:2258] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:15.031033 31307 tablet_replica.cc:330] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb: stopping tablet replica
I20260926 20:40:15.031251 31307 raft_consensus.cc:2229] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:15.031561 31307 raft_consensus.cc:2258] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:15.032517 31308 ts_tablet_manager.cc:1905] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:15.033533 31307 ts_tablet_manager.cc:1905] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:15.039567 31308 ts_tablet_manager.cc:1918] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:15.039722 31308 log.cc:1196] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/8eba692ee12d4bd58937d83c8d43aaa1
I20260926 20:40:15.040275 31308 ts_tablet_manager.cc:1939] T 8eba692ee12d4bd58937d83c8d43aaa1 P e01a9a75c042415dacc31c4cfb89a56b: Deleting consensus metadata
I20260926 20:40:15.040243 31307 ts_tablet_manager.cc:1918] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:15.040585 31307 log.cc:1196] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/8eba692ee12d4bd58937d83c8d43aaa1
I20260926 20:40:15.041124 31307 ts_tablet_manager.cc:1939] T 8eba692ee12d4bd58937d83c8d43aaa1 P a08386306b2d4fbc83729667f83d09fb: Deleting consensus metadata
I20260926 20:40:15.041739 30703 catalog_manager.cc:4841] TS e01a9a75c042415dacc31c4cfb89a56b (127.23.44.195:33263): tablet 8eba692ee12d4bd58937d83c8d43aaa1 (table test-table [id=447517ec61524d16be0ef981ed182fe7]) successfully deleted
I20260926 20:40:15.042402 30702 catalog_manager.cc:4841] TS a08386306b2d4fbc83729667f83d09fb (127.23.44.194:39483): tablet 8eba692ee12d4bd58937d83c8d43aaa1 (table test-table [id=447517ec61524d16be0ef981ed182fe7]) successfully deleted
W20260926 20:40:15.042678 30703 catalog_manager.cc:4808] TS e01a9a75c042415dacc31c4cfb89a56b (127.23.44.195:33263): delete failed for tablet 8eba692ee12d4bd58937d83c8d43aaa1 because the tablet was not found. No further retry: Not found: Tablet not found: 8eba692ee12d4bd58937d83c8d43aaa1
I20260926 20:40:15.049721 31309 tablet_replica.cc:330] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1: stopping tablet replica
I20260926 20:40:15.050096 31309 raft_consensus.cc:2229] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:15.050631 31309 raft_consensus.cc:2258] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:15.056542 31309 ts_tablet_manager.cc:1905] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:15.064090 31309 ts_tablet_manager.cc:1918] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:15.064311 31309 log.cc:1196] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/8eba692ee12d4bd58937d83c8d43aaa1
I20260926 20:40:15.065064 31309 ts_tablet_manager.cc:1939] T 8eba692ee12d4bd58937d83c8d43aaa1 P a32161d75a4447f7a69c09e3024251f1: Deleting consensus metadata
I20260926 20:40:15.066543 30700 catalog_manager.cc:4841] TS a32161d75a4447f7a69c09e3024251f1 (127.23.44.193:35693): tablet 8eba692ee12d4bd58937d83c8d43aaa1 (table test-table [id=447517ec61524d16be0ef981ed182fe7]) successfully deleted
W20260926 20:40:15.067315 30700 catalog_manager.cc:4808] TS a32161d75a4447f7a69c09e3024251f1 (127.23.44.193:35693): delete failed for tablet 8eba692ee12d4bd58937d83c8d43aaa1 because the tablet was not found. No further retry: Not found: Tablet not found: 8eba692ee12d4bd58937d83c8d43aaa1
I20260926 20:40:15.072866 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:15.026992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57496 (local address 127.23.44.193:35693)
0926 20:40:15.027177 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:15.027184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:15.031186 (+ 4002us) server_negotiation.cc:407] Connection header received
0926 20:40:15.031422 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:15.031428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:15.031536 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:15.031655 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:15.052525 (+ 20870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:15.053781 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:15.060662 (+ 6881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:15.061400 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:15.061539 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:15.065542 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:15.065564 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:15.065591 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:15.065641 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:15.069448 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:15.070234 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:15.070250 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:15.070252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:15.070381 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:15.070783 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:15.070788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:15.070792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:15.071193 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:15.071565 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:15.071922 (+ 357us) server_negotiation.cc:299] Negotiation successful
0926 20:40:15.072526 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:40:15.076932 30891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:15.203183 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 30746
I20260926 20:40:15.226580 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 30908
I20260926 20:40:15.241807 31277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:15.227425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51234 (local address 127.23.44.194:39483)
0926 20:40:15.227592 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:15.227599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:15.227685 (+ 86us) server_negotiation.cc:407] Connection header received
0926 20:40:15.227888 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:15.227894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:15.227973 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:15.228079 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:15.229383 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:15.230164 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:15.231391 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:15.231789 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:15.231922 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:15.234571 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:15.234584 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:15.234605 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:15.234640 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:15.238276 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:15.239330 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:15.239344 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:15.239348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:15.239542 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:15.240019 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:15.240025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:15.240028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:15.240413 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:15.241065 (+ 652us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:15.241224 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 20:40:15.241473 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 20:40:15.242655 31037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:15.274607 31313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:15.273761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:50148)
0926 20:40:15.274256 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:15.274367 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":256,"threads_started":1}
I20260926 20:40:15.276335 31313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:15.275857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:41983 (local address 127.0.0.1:50164)
0926 20:40:15.276053 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:15.276164 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 20:40:15.340711 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 30908
I20260926 20:40:15.360513 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 31055
I20260926 20:40:15.377813 31278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:15.361365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58602 (local address 127.23.44.195:33263)
0926 20:40:15.361612 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:15.361618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:15.361640 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:40:15.361741 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:15.361749 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:15.361841 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:15.361967 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:15.363367 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:15.364544 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:15.366035 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:15.366727 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:15.366878 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:15.369821 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:15.369839 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:15.369859 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:15.369908 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:15.374125 (+ 4217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:15.375009 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:15.375019 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:15.375022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:15.375209 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:15.375600 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:15.375607 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:15.375610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:15.376145 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:15.376466 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:15.376892 (+ 426us) server_negotiation.cc:299] Negotiation successful
0926 20:40:15.377242 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 20:40:15.378594 31178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:15.507146 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 31055
I20260926 20:40:15.530328 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 30388
I20260926 20:40:15.531563 30555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:15.649140 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 30388
I20260926 20:40:15.670563 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 30489
I20260926 20:40:15.671666 30747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:15.792351 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 30489
2026-09-26T20:40:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10248 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:40:15 dist-test-slave-4d5t krb5kdc[31320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:40:15 dist-test-slave-4d5t krb5kdc[31320](info): set up 2 sockets
Sep 26 20:40:15 dist-test-slave-4d5t krb5kdc[31320](info): commencing operation
krb5kdc: starting...
W20260926 20:40:16.910154 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:40:17 dist-test-slave-4d5t krb5kdc[31320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:40:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:40:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.254.keytab.
Entry for principal oryx/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.254.keytab.
I20260926 20:40:17.156934 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:39411
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.254.keytab
--principal=oryx/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:39411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:17.496433 31333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:17.496899 31333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:17.497064 31333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:17.504753 31333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:40:17.504915 31333 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 20:40:17.504988 31333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:17.505088 31333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:17.505185 31333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:40:17.505280 31333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:40:17.517009 31333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:39411
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.254.keytab
--principal=oryx/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:39411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:17.518476 31333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:17.519966 31333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:17.532572 31338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:17.532799 31339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:17.532799 31341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:17.534003 31333 server_base.cc:1034] running on GCE node
I20260926 20:40:17.535115 31333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:17.536983 31333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:17.538269 31333 hybrid_clock.cc:648] HybridClock initialized: now 1790455217538204 us; error 56 us; skew 500 ppm
I20260926 20:40:17.538650 31333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:17 dist-test-slave-4d5t krb5kdc[31320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455217, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:17.542671 31333 init.cc:377] Logged in from keytab as oryx/127.23.44.254@KRBTEST.COM (short username oryx)
I20260926 20:40:17.544636 31333 webserver.cc:460] Webserver started at http://127.23.44.254:40243/ using document root <none> and password file <none>
I20260926 20:40:17.545413 31333 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:17.545552 31333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:17.545892 31333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:17.549358 31333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "7c19d17dfa9e41269f5f69adce776c6c"
format_stamp: "Formatted at 2026-09-26 20:40:17 on dist-test-slave-4d5t"
server_key: "b9aab7be9652f5473362ad1fdee4557f"
server_key_iv: "49bcdd0918ec32f295cf7af49c42e38a"
server_key_version: "encryptionkey@0"
I20260926 20:40:17.550220 31333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "7c19d17dfa9e41269f5f69adce776c6c"
format_stamp: "Formatted at 2026-09-26 20:40:17 on dist-test-slave-4d5t"
server_key: "b9aab7be9652f5473362ad1fdee4557f"
server_key_iv: "49bcdd0918ec32f295cf7af49c42e38a"
server_key_version: "encryptionkey@0"
I20260926 20:40:17.556119 31333 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 20:40:17.560272 31348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:17.561903 31333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:40:17.562145 31333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "7c19d17dfa9e41269f5f69adce776c6c"
format_stamp: "Formatted at 2026-09-26 20:40:17 on dist-test-slave-4d5t"
server_key: "b9aab7be9652f5473362ad1fdee4557f"
server_key_iv: "49bcdd0918ec32f295cf7af49c42e38a"
server_key_version: "encryptionkey@0"
I20260926 20:40:17.562414 31333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:17.599059 31333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:17.603785 31333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:17.604080 31333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:17.627585 31409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:39411 every 8 connection(s)
I20260926 20:40:17.627610 31333 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:39411
I20260926 20:40:17.629016 31333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:40:17.632426 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 31333
I20260926 20:40:17.632679 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:40:17.633535 23731 external_mini_cluster.cc:1428] Setting key 93809d94bc78df6d19488735f4ce7f55
I20260926 20:40:17.635082 31410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:40:17 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
I20260926 20:40:17.650862 31410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c: Bootstrap starting.
I20260926 20:40:17.654639 31410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:17.655841 31410 log.cc:824] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:17.659197 31410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c: No bootstrap required, opened a new log
I20260926 20:40:17.659215 31413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:17.635797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47420 (local address 127.23.44.254:39411)
0926 20:40:17.636711 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:17.636725 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:17.636769 (+ 44us) server_negotiation.cc:407] Connection header received
0926 20:40:17.638515 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:17.638539 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:17.639096 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:17.639566 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:17.641237 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:17.642672 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:17.643913 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:17.644702 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:17.644900 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:17.649648 (+ 4748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:17.649659 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:17.649683 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:17.649866 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:17.653962 (+ 4096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:17.654982 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:17.654990 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:17.654999 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:17.655225 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:17.655659 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:17.655672 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:17.655674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:17.656764 (+ 1090us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:17.657177 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:17.657758 (+ 581us) server_negotiation.cc:299] Negotiation successful
0926 20:40:17.658214 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":396,"threads_started":1}
I20260926 20:40:17.668172 31410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c19d17dfa9e41269f5f69adce776c6c" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39411 } }
I20260926 20:40:17.668658 31410 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:17.668779 31410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c19d17dfa9e41269f5f69adce776c6c, State: Initialized, Role: FOLLOWER
I20260926 20:40:17.669364 31410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c19d17dfa9e41269f5f69adce776c6c" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39411 } }
I20260926 20:40:17.669628 31410 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:40:17.669783 31410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:40:17.669978 31410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:17.672398 31410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c19d17dfa9e41269f5f69adce776c6c" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39411 } }
I20260926 20:40:17.673038 31410 leader_election.cc:302] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c19d17dfa9e41269f5f69adce776c6c; no voters:
I20260926 20:40:17.673671 31410 leader_election.cc:288] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:40:17.673825 31415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:17.674568 31415 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [term 1 LEADER]: Becoming Leader. State: Replica: 7c19d17dfa9e41269f5f69adce776c6c, State: Running, Role: LEADER
I20260926 20:40:17.675154 31415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c19d17dfa9e41269f5f69adce776c6c" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39411 } }
I20260926 20:40:17.676008 31410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:40:17.678119 31417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c19d17dfa9e41269f5f69adce776c6c. Latest consensus state: current_term: 1 leader_uuid: "7c19d17dfa9e41269f5f69adce776c6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c19d17dfa9e41269f5f69adce776c6c" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39411 } } }
I20260926 20:40:17.678114 31416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c19d17dfa9e41269f5f69adce776c6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c19d17dfa9e41269f5f69adce776c6c" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39411 } } }
I20260926 20:40:17.678565 31417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:17.678598 31416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:17.680464 31420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:40:17.688246 31420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:40:17.699067 31420 catalog_manager.cc:1347] Generated new cluster ID: 60b006c5d1bb4159a40d306a4461e110
I20260926 20:40:17.699209 31420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:40:17.723562 31420 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:40:17.725069 31420 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:40:17.735491 31420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c: Generated new TSK 0
I20260926 20:40:17.736783 31420 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab.
Entry for principal oryx/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab.
I20260926 20:40:17.802667 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39411
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab
--principal=oryx/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:18.156965 31436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:18.157346 31436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:18.157539 31436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:18.165068 31436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:18.165239 31436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:18.165380 31436 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:40:18.165524 31436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:40:18.175673 31436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab
--principal=oryx/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.44.254:39411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:18.177212 31436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:18.178646 31436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:18.193154 31441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:18.193168 31442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:18.193025 31444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:18.194335 31436 server_base.cc:1034] running on GCE node
I20260926 20:40:18.195020 31436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:18.196048 31436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:18.197351 31436 hybrid_clock.cc:648] HybridClock initialized: now 1790455218197274 us; error 40 us; skew 500 ppm
I20260926 20:40:18.197680 31436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:18 dist-test-slave-4d5t krb5kdc[31320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455218, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:18.207728 31436 init.cc:377] Logged in from keytab as oryx/127.23.44.193@KRBTEST.COM (short username oryx)
I20260926 20:40:18.209614 31436 webserver.cc:460] Webserver started at http://127.23.44.193:45487/ using document root <none> and password file <none>
I20260926 20:40:18.210393 31436 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:18.210556 31436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:18.210923 31436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:18.214179 31436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7"
format_stamp: "Formatted at 2026-09-26 20:40:18 on dist-test-slave-4d5t"
server_key: "630ba15da575ad8656cffd324ce52d02"
server_key_iv: "11f173c9342898234758d6f5549498cf"
server_key_version: "encryptionkey@0"
I20260926 20:40:18.215126 31436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7"
format_stamp: "Formatted at 2026-09-26 20:40:18 on dist-test-slave-4d5t"
server_key: "630ba15da575ad8656cffd324ce52d02"
server_key_iv: "11f173c9342898234758d6f5549498cf"
server_key_version: "encryptionkey@0"
I20260926 20:40:18.221046 31436 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 20:40:18.224756 31451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:18.226154 31436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:40:18.226346 31436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7"
format_stamp: "Formatted at 2026-09-26 20:40:18 on dist-test-slave-4d5t"
server_key: "630ba15da575ad8656cffd324ce52d02"
server_key_iv: "11f173c9342898234758d6f5549498cf"
server_key_version: "encryptionkey@0"
I20260926 20:40:18.226590 31436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:18.245504 31436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:18.250233 31436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:18.250519 31436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:18.253315 31436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:18.253430 31436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:18.253598 31436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:18.253680 31436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:40:18 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455218, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
I20260926 20:40:18.279958 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:18.253915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:58097 (local address 127.23.44.254:39411)
0926 20:40:18.254582 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:18.254591 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:18.256335 (+ 1744us) server_negotiation.cc:407] Connection header received
0926 20:40:18.259388 (+ 3053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:18.259397 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:18.259514 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:18.259694 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:18.262835 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.263896 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.266387 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.267098 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.267250 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:18.272298 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:18.272308 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:18.272316 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:18.272364 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:18.276529 (+ 4165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:18.277488 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:18.277494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:18.277498 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:18.277627 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:18.278085 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:18.278089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:18.278093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:18.278365 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:18.278525 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:18.279360 (+ 835us) server_negotiation.cc:299] Negotiation successful
0926 20:40:18.279597 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":415,"threads_started":1}
I20260926 20:40:18.281287 31461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:18.254634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39411 (local address 127.23.44.193:58097)
0926 20:40:18.256040 (+ 1406us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:18.256122 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:18.258975 (+ 2853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:18.260055 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:18.260066 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:18.261475 (+ 1409us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:18.262564 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:18.262603 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.264077 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.264082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:18.266211 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:18.266224 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.267366 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.267375 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:18.267653 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:18.268531 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:18.268582 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:18.272004 (+ 3422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:18.276714 (+ 4710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:18.276724 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:18.276755 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:18.277338 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:18.277739 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:18.277744 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:18.277748 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:18.277963 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:18.278508 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:18.278517 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:18.278990 (+ 473us) client_negotiation.cc:769] Sending connection context
0926 20:40:18.279257 (+ 267us) client_negotiation.cc:240] Negotiation successful
0926 20:40:18.279817 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"thread_start_us":722,"threads_started":1}
I20260926 20:40:18.305125 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:18.288311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39411 (local address 127.23.44.193:59045)
0926 20:40:18.288988 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:18.289012 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:18.289198 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:18.289615 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:18.289619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:18.290298 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:18.290897 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:18.290910 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.292288 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.292294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:18.293913 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:18.293934 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.295195 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.295204 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:18.295405 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:18.296128 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:18.296160 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:18.297118 (+ 958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:18.301412 (+ 4294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:18.301417 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:18.301425 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:18.302065 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:18.302557 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:18.302561 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:18.302564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:18.302747 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:18.303589 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:18.303602 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:18.303853 (+ 251us) client_negotiation.cc:769] Sending connection context
0926 20:40:18.303971 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 20:40:18.304453 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":362,"threads_started":1}
I20260926 20:40:18.305485 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:18.288311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:59045 (local address 127.23.44.254:39411)
0926 20:40:18.288606 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:18.288614 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:18.289111 (+ 497us) server_negotiation.cc:407] Connection header received
0926 20:40:18.289359 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:18.289367 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:18.289461 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:18.289576 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:18.291055 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.292137 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.294079 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.294886 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.295054 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:18.297322 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:18.297330 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:18.297336 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:18.297374 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:18.301235 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:18.302249 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:18.302256 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:18.302260 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:18.302424 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:18.303054 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:18.303061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:18.303064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:18.303377 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:18.304689 (+ 1312us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:18.304848 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 20:40:18.305102 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 20:40:18.307011 31436 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:33983
I20260926 20:40:18.307034 31571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:33983 every 8 connection(s)
I20260926 20:40:18.308530 31436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:40:18.309899 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 31436
I20260926 20:40:18.310109 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:40:18.310968 23731 external_mini_cluster.cc:1428] Setting key 49218b778f5f87ac7ce5d71866cf0728
I20260926 20:40:18.310956 31572 heartbeater.cc:347] Connected to a master server at 127.23.44.254:39411
I20260926 20:40:18.311179 31572 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:18.311931 31572 heartbeater.cc:511] Master 127.23.44.254:39411 requested a full tablet report, sending...
I20260926 20:40:18.313897 31365 ts_manager.cc:194] Registered new tserver with Master: 69a1f31efdcc44e7a1dd36ea8e70c6d7 (127.23.44.193:33983)
I20260926 20:40:18.315315 31365 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:58097
WARNING: no policy specified for oryx/127.23.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab.
Entry for principal oryx/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab.
I20260926 20:40:18.360352 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39411
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab
--principal=oryx/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:18.698547 31576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:18.698962 31576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:18.699158 31576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:18.707288 31576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:18.707472 31576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:18.707638 31576 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:40:18.707785 31576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:40:18.718498 31576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab
--principal=oryx/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.44.254:39411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:18.719916 31576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:18.721374 31576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:18.736382 31584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:18.736475 31581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:18.736541 31582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:18.737289 31576 server_base.cc:1034] running on GCE node
I20260926 20:40:18.737867 31576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:18.738847 31576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:18.740091 31576 hybrid_clock.cc:648] HybridClock initialized: now 1790455218740030 us; error 32 us; skew 500 ppm
I20260926 20:40:18.740458 31576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:18 dist-test-slave-4d5t krb5kdc[31320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455218, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:18.744349 31576 init.cc:377] Logged in from keytab as oryx/127.23.44.194@KRBTEST.COM (short username oryx)
I20260926 20:40:18.746219 31576 webserver.cc:460] Webserver started at http://127.23.44.194:36803/ using document root <none> and password file <none>
I20260926 20:40:18.747015 31576 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:18.747112 31576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:18.747486 31576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:18.750597 31576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "c305b130115a46fd8a0dd27815c7dbbc"
format_stamp: "Formatted at 2026-09-26 20:40:18 on dist-test-slave-4d5t"
server_key: "1207ca9cd9746e10caf1773a255bb717"
server_key_iv: "981849b35738b25ad6ccfea4cf1ccd13"
server_key_version: "encryptionkey@0"
I20260926 20:40:18.751935 31576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "c305b130115a46fd8a0dd27815c7dbbc"
format_stamp: "Formatted at 2026-09-26 20:40:18 on dist-test-slave-4d5t"
server_key: "1207ca9cd9746e10caf1773a255bb717"
server_key_iv: "981849b35738b25ad6ccfea4cf1ccd13"
server_key_version: "encryptionkey@0"
I20260926 20:40:18.759779 31576 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260926 20:40:18.764027 31591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:18.766299 31576 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:40:18.766664 31576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "c305b130115a46fd8a0dd27815c7dbbc"
format_stamp: "Formatted at 2026-09-26 20:40:18 on dist-test-slave-4d5t"
server_key: "1207ca9cd9746e10caf1773a255bb717"
server_key_iv: "981849b35738b25ad6ccfea4cf1ccd13"
server_key_version: "encryptionkey@0"
I20260926 20:40:18.766974 31576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:18.790061 31576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:18.795507 31576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:18.795854 31576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:18.799125 31576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:18.799250 31576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:18.799422 31576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:18.799504 31576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:40:18 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455218, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
I20260926 20:40:18.833946 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:18.799760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:60091 (local address 127.23.44.254:39411)
0926 20:40:18.799990 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:18.799995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:18.801740 (+ 1745us) server_negotiation.cc:407] Connection header received
0926 20:40:18.805010 (+ 3270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:18.805020 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:18.805129 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:18.805230 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:18.809169 (+ 3939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.809915 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.815141 (+ 5226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.815960 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.816134 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:18.822650 (+ 6516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:18.822661 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:18.822670 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:18.822732 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:18.827102 (+ 4370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:18.829063 (+ 1961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:18.829072 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:18.829079 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:18.829270 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:18.829991 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:18.829998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:18.830004 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:18.830423 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:18.832769 (+ 2346us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:18.832985 (+ 216us) server_negotiation.cc:299] Negotiation successful
0926 20:40:18.833398 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 20:40:18.834324 31600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:18.800460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39411 (local address 127.23.44.194:60091)
0926 20:40:18.801488 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:18.801561 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:18.804539 (+ 2978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:18.805924 (+ 1385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:18.805942 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:18.807064 (+ 1122us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:18.808887 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:18.808928 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.810293 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.810303 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:18.814901 (+ 4598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:18.814927 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.816312 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.816327 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:18.816792 (+ 465us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:18.817867 (+ 1075us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:18.817912 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:18.822271 (+ 4359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:18.827407 (+ 5136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:18.827420 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:18.827449 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:18.828670 (+ 1221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:18.829452 (+ 782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:18.829461 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:18.829465 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:18.829793 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:18.830594 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:18.830606 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:18.831237 (+ 631us) client_negotiation.cc:769] Sending connection context
0926 20:40:18.831614 (+ 377us) client_negotiation.cc:240] Negotiation successful
0926 20:40:18.832373 (+ 759us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":399,"threads_started":1}
I20260926 20:40:18.847323 31576 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:45199
I20260926 20:40:18.847380 31710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:45199 every 8 connection(s)
I20260926 20:40:18.848850 31576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:40:18.852301 31711 heartbeater.cc:347] Connected to a master server at 127.23.44.254:39411
I20260926 20:40:18.852658 31711 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:18.853649 31711 heartbeater.cc:511] Master 127.23.44.254:39411 requested a full tablet report, sending...
I20260926 20:40:18.854840 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 31576
I20260926 20:40:18.855036 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:40:18.855332 31365 ts_manager.cc:194] Registered new tserver with Master: c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194:45199)
I20260926 20:40:18.855640 23731 external_mini_cluster.cc:1428] Setting key 382de0b6f35e443ae0db5d100f719d3d
I20260926 20:40:18.856348 31365 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:60091
I20260926 20:40:18.860992 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:18.843760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:54431 (local address 127.23.44.254:39411)
0926 20:40:18.843950 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:18.843955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:18.844712 (+ 757us) server_negotiation.cc:407] Connection header received
0926 20:40:18.845030 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:18.845036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:18.845111 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:18.845215 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:18.846537 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.847683 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.849672 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.850146 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.850302 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:18.853649 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:18.853655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:18.853658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:18.853698 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:18.857523 (+ 3825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:18.858507 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:18.858515 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:18.858521 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:18.858696 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:18.859163 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:18.859168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:18.859172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:18.859606 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:18.859763 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:18.860332 (+ 569us) server_negotiation.cc:299] Negotiation successful
0926 20:40:18.860613 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:40:18.861101 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:18.843829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39411 (local address 127.23.44.194:54431)
0926 20:40:18.844543 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:18.844615 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:18.844854 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:18.845246 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:18.845252 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:18.845736 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:18.846335 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:18.846351 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.847841 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.847844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:18.849402 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:18.849430 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:18.850497 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:18.850512 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:18.850757 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:18.851866 (+ 1109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:18.851904 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:18.853363 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:18.857720 (+ 4357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:18.857726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:18.857731 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:18.858351 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:18.858860 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:18.858865 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:18.858869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:18.859026 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:18.859850 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:18.859855 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:18.860060 (+ 205us) client_negotiation.cc:769] Sending connection context
0926 20:40:18.860221 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 20:40:18.860601 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":433,"threads_started":1}
WARNING: no policy specified for oryx/127.23.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab.
Entry for principal oryx/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab.
I20260926 20:40:18.912974 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39411
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab
--principal=oryx/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:19.255532 31715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:19.255833 31715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:19.255937 31715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:19.264340 31715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:19.264478 31715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:19.264649 31715 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 20:40:19.264751 31715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:40:19.277209 31715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab
--principal=oryx/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.44.254:39411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:19.278995 31715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:19.281050 31715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:19.296164 31720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:19.296096 31723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:19.296133 31721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:19.298174 31715 server_base.cc:1034] running on GCE node
I20260926 20:40:19.298843 31715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:19.299852 31715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:19.301120 31715 hybrid_clock.cc:648] HybridClock initialized: now 1790455219301058 us; error 43 us; skew 500 ppm
I20260926 20:40:19.301469 31715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:19 dist-test-slave-4d5t krb5kdc[31320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455219, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:19.305145 31715 init.cc:377] Logged in from keytab as oryx/127.23.44.195@KRBTEST.COM (short username oryx)
I20260926 20:40:19.306862 31715 webserver.cc:460] Webserver started at http://127.23.44.195:33835/ using document root <none> and password file <none>
I20260926 20:40:19.307591 31715 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:19.307724 31715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:19.308045 31715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:19.311847 31715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "fea36b613dc84692831b4ea0079b8dd8"
format_stamp: "Formatted at 2026-09-26 20:40:19 on dist-test-slave-4d5t"
server_key: "baf4670bfac9bde5d05bbff84a71b180"
server_key_iv: "6cc4a1ac4f2389522a728c52a6a5c4d0"
server_key_version: "encryptionkey@0"
I20260926 20:40:19.312827 31715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "fea36b613dc84692831b4ea0079b8dd8"
format_stamp: "Formatted at 2026-09-26 20:40:19 on dist-test-slave-4d5t"
server_key: "baf4670bfac9bde5d05bbff84a71b180"
server_key_iv: "6cc4a1ac4f2389522a728c52a6a5c4d0"
server_key_version: "encryptionkey@0"
I20260926 20:40:19.318742 31572 heartbeater.cc:503] Master 127.23.44.254:39411 was elected leader, sending a full tablet report...
I20260926 20:40:19.319016 31715 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 20:40:19.323400 31730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:19.325088 31715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:40:19.325299 31715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "fea36b613dc84692831b4ea0079b8dd8"
format_stamp: "Formatted at 2026-09-26 20:40:19 on dist-test-slave-4d5t"
server_key: "baf4670bfac9bde5d05bbff84a71b180"
server_key_iv: "6cc4a1ac4f2389522a728c52a6a5c4d0"
server_key_version: "encryptionkey@0"
I20260926 20:40:19.325559 31715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:19.355034 31715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:19.359720 31715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:19.360061 31715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:19.363071 31715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:19.363152 31715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:19.363320 31715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:19.363379 31715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:40:19 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455219, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
I20260926 20:40:19.391279 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:19.363791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:44985 (local address 127.23.44.254:39411)
0926 20:40:19.364332 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:19.364339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:19.365832 (+ 1493us) server_negotiation.cc:407] Connection header received
0926 20:40:19.368105 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:19.368110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:19.368211 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:19.368357 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:19.371177 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:19.372210 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:19.375524 (+ 3314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:19.376367 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:19.376605 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:19.381911 (+ 5306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:19.381922 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:19.381929 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:19.381990 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:19.386170 (+ 4180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:19.387566 (+ 1396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:19.387574 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:19.387580 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:19.387761 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:19.388452 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:19.388460 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:19.388466 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:19.388962 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:19.389145 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:19.390431 (+ 1286us) server_negotiation.cc:299] Negotiation successful
0926 20:40:19.390744 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":310,"threads_started":1}
I20260926 20:40:19.392954 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:19.364504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39411 (local address 127.23.44.195:44985)
0926 20:40:19.365483 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:19.365529 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:19.367789 (+ 2260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:19.368697 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:19.368709 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:19.369773 (+ 1064us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:19.370899 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:19.370936 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:19.372387 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:19.372396 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:19.375257 (+ 2861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:19.375283 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:19.376767 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:19.376777 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:19.377084 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:19.378171 (+ 1087us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:19.378207 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:19.381485 (+ 3278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:19.386413 (+ 4928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:19.386427 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:19.386458 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:19.387355 (+ 897us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:19.387918 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:19.387926 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:19.387932 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:19.388281 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:19.389140 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:19.389152 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:19.389841 (+ 689us) client_negotiation.cc:769] Sending connection context
0926 20:40:19.390265 (+ 424us) client_negotiation.cc:240] Negotiation successful
0926 20:40:19.391027 (+ 762us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":386,"threads_started":1}
I20260926 20:40:19.416864 31715 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:41185
I20260926 20:40:19.416916 31850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:41185 every 8 connection(s)
I20260926 20:40:19.418274 31715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:40:19.418605 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:19.403252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:34703 (local address 127.23.44.254:39411)
0926 20:40:19.403495 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:19.403504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:19.403649 (+ 145us) server_negotiation.cc:407] Connection header received
0926 20:40:19.403795 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:19.403803 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:19.403911 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:19.404031 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:19.405443 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:19.406705 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:19.408245 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:19.409083 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:19.409236 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:19.411655 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:19.411665 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:19.411673 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:19.411725 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:19.415228 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:19.415951 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:19.415959 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:19.415964 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:19.416149 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:19.416515 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:19.416522 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:19.416525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:19.416968 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:19.417176 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:19.418000 (+ 824us) server_negotiation.cc:299] Negotiation successful
0926 20:40:19.418238 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 20:40:19.418749 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:19.402638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39411 (local address 127.23.44.195:34703)
0926 20:40:19.403527 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:19.403548 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:19.403687 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:19.404099 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:19.404106 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:19.404868 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:19.405222 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:19.405239 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:19.406884 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:19.406893 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:19.408099 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:19.408111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:19.409321 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:19.409328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:19.409450 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:19.410185 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:19.410213 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:19.411414 (+ 1201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:19.415396 (+ 3982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:19.415401 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:19.415406 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:19.415820 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:19.416251 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:19.416256 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:19.416260 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:19.416393 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:19.417134 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:19.417139 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:19.417469 (+ 330us) client_negotiation.cc:769] Sending connection context
0926 20:40:19.417795 (+ 326us) client_negotiation.cc:240] Negotiation successful
0926 20:40:19.418280 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":596,"threads_started":1}
I20260926 20:40:19.422346 31851 heartbeater.cc:347] Connected to a master server at 127.23.44.254:39411
I20260926 20:40:19.422634 31851 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:19.423518 31851 heartbeater.cc:511] Master 127.23.44.254:39411 requested a full tablet report, sending...
I20260926 20:40:19.425181 31365 ts_manager.cc:194] Registered new tserver with Master: fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185)
I20260926 20:40:19.426011 31365 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:44985
I20260926 20:40:19.428654 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 31715
I20260926 20:40:19.428839 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:40:19.429337 23731 external_mini_cluster.cc:1428] Setting key 90de4d21d0e397cffa7195d2605b9baa
I20260926 20:40:19.432911 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 20:40:19 dist-test-slave-4d5t krb5kdc[31320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455219, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 20:40:19 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:19 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:19.860710 31711 heartbeater.cc:503] Master 127.23.44.254:39411 was elected leader, sending a full tablet report...
I20260926 20:40:20.428647 31851 heartbeater.cc:503] Master 127.23.44.254:39411 was elected leader, sending a full tablet report...
W20260926 20:40:20.609673 31859 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.125s user 0.004s sys 0.011s
W20260926 20:40:20.610870 31739 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:19.478269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47422 (local address 127.23.44.254:39411)
0926 20:40:19.478506 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:19.478511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:19.478641 (+ 130us) server_negotiation.cc:407] Connection header received
0926 20:40:19.478847 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:19.478852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:19.478916 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:19.479006 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:19.480516 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:19.481721 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:19.483409 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:19.483997 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:19.484120 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:20.610628 (+1126508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47422: BlockingRecv error: recv got EOF from 127.0.0.1:47422 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 26 20:40:20 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455219, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
I20260926 20:40:20.632376 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:20.615085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47434 (local address 127.23.44.254:39411)
0926 20:40:20.615314 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:20.615320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:20.615629 (+ 309us) server_negotiation.cc:407] Connection header received
0926 20:40:20.615784 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:20.615790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:20.615890 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:20.616039 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:20.617483 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.618240 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.619826 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.620336 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.620450 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:20.625511 (+ 5061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:20.625518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:20.625521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:20.625568 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:20.628419 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:20.629445 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:20.629451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:20.629454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:20.629835 (+ 381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:20.630375 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:20.630380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:20.630382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:20.631085 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:20.631299 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:20.631783 (+ 484us) server_negotiation.cc:299] Negotiation successful
0926 20:40:20.631985 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 20:40:20.637169 31365 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:40:20.645526 31365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:40:20.671515 31872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:20.660237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:33983 (local address 127.0.0.1:49024)
0926 20:40:20.662222 (+ 1985us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:20.662248 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:20.662377 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:20.663470 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:20.663478 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:20.663531 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:20.664007 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:20.664019 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.665917 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.665922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:20.668407 (+ 2485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:20.668420 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.670331 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.670339 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:20.670585 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:20.670656 (+ 71us) client_negotiation.cc:769] Sending connection context
0926 20:40:20.670837 (+ 181us) client_negotiation.cc:240] Negotiation successful
0926 20:40:20.671055 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1817,"thread_start_us":1696,"threads_started":1}
I20260926 20:40:20.671674 31873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:20.660604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49024 (local address 127.23.44.193:33983)
0926 20:40:20.661913 (+ 1309us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:20.661922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:20.662532 (+ 610us) server_negotiation.cc:407] Connection header received
0926 20:40:20.662645 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:20.662656 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:20.663001 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:20.663218 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:20.664206 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.665725 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.668606 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.670076 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.670231 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:20.670451 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:20.671113 (+ 662us) server_negotiation.cc:299] Negotiation successful
0926 20:40:20.671296 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1102,"thread_start_us":855,"threads_started":1}
I20260926 20:40:20.671870 31870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:20.658987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:41185 (local address 127.0.0.1:43712)
0926 20:40:20.659976 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:20.660036 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:20.660225 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:20.662447 (+ 2222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:20.662459 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:20.662502 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:20.663040 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:20.663055 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.665343 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.665350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:20.668779 (+ 3429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:20.668796 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.670838 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.670845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:20.671016 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:20.671048 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 20:40:20.671213 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 20:40:20.671368 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":105,"thread_start_us":450,"threads_started":1}
I20260926 20:40:20.672650 31871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:20.659141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43712 (local address 127.23.44.195:41185)
0926 20:40:20.661431 (+ 2290us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:20.661447 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:20.661501 (+ 54us) server_negotiation.cc:407] Connection header received
0926 20:40:20.661673 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:20.661682 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:20.662054 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:20.663290 (+ 1236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:20.663536 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.665187 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.669004 (+ 3817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.670480 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.670776 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:20.670972 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:20.671967 (+ 995us) server_negotiation.cc:299] Negotiation successful
0926 20:40:20.672179 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1967,"thread_start_us":1848,"threads_started":1}
I20260926 20:40:20.674371 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:20.663252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:45199 (local address 127.0.0.1:36190)
0926 20:40:20.664806 (+ 1554us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:20.664832 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:20.664970 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:20.665568 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:20.665574 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:20.665609 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:20.666205 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:20.666236 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.668720 (+ 2484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.668727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:20.670439 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:20.670451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.673567 (+ 3116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.673573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:20.673745 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:20.673792 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 20:40:20.673903 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 20:40:20.674046 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1422,"thread_start_us":1241,"threads_started":1}
I20260926 20:40:20.674386 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:20.662524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36190 (local address 127.23.44.194:45199)
0926 20:40:20.664289 (+ 1765us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:20.664299 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:20.664973 (+ 674us) server_negotiation.cc:407] Connection header received
0926 20:40:20.665095 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:20.665102 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:20.665345 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:20.665552 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:20.666645 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.668411 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.670678 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.673335 (+ 2657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.673508 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:20.673726 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:20.673943 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 20:40:20.674105 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1580,"thread_start_us":1407,"threads_started":1}
I20260926 20:40:20.676910 31502 tablet_service.cc:1467] Processing CreateTablet for tablet 90aaa3a1cdcf4458b6ff25507eb8cbe5 (DEFAULT_TABLE table=test-table [id=19927618ed0048dda15547bf41ff3d0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:20.676874 31781 tablet_service.cc:1467] Processing CreateTablet for tablet 90aaa3a1cdcf4458b6ff25507eb8cbe5 (DEFAULT_TABLE table=test-table [id=19927618ed0048dda15547bf41ff3d0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:20.678151 31641 tablet_service.cc:1467] Processing CreateTablet for tablet 90aaa3a1cdcf4458b6ff25507eb8cbe5 (DEFAULT_TABLE table=test-table [id=19927618ed0048dda15547bf41ff3d0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:20.678476 31781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90aaa3a1cdcf4458b6ff25507eb8cbe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:20.678489 31502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90aaa3a1cdcf4458b6ff25507eb8cbe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:20.679144 31641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90aaa3a1cdcf4458b6ff25507eb8cbe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:20.692790 31876 tablet_bootstrap.cc:492] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc: Bootstrap starting.
I20260926 20:40:20.695116 31877 tablet_bootstrap.cc:492] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8: Bootstrap starting.
I20260926 20:40:20.697160 31878 tablet_bootstrap.cc:492] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Bootstrap starting.
I20260926 20:40:20.698989 31876 tablet_bootstrap.cc:654] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:20.700389 31877 tablet_bootstrap.cc:654] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:20.700431 31878 tablet_bootstrap.cc:654] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:20.700546 31876 log.cc:824] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:20.701802 31877 log.cc:824] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:20.702064 31878 log.cc:824] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:20.704905 31876 tablet_bootstrap.cc:492] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc: No bootstrap required, opened a new log
I20260926 20:40:20.705053 31877 tablet_bootstrap.cc:492] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8: No bootstrap required, opened a new log
I20260926 20:40:20.705268 31876 ts_tablet_manager.cc:1397] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260926 20:40:20.705377 31877 ts_tablet_manager.cc:1397] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260926 20:40:20.705696 31878 tablet_bootstrap.cc:492] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: No bootstrap required, opened a new log
I20260926 20:40:20.705996 31878 ts_tablet_manager.cc:1397] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260926 20:40:20.715169 31877 raft_consensus.cc:348] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } }
I20260926 20:40:20.715682 31877 raft_consensus.cc:374] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:20.715852 31877 raft_consensus.cc:729] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea36b613dc84692831b4ea0079b8dd8, State: Initialized, Role: FOLLOWER
I20260926 20:40:20.716698 31877 consensus_queue.cc:260] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } }
I20260926 20:40:20.717965 31878 raft_consensus.cc:348] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } }
I20260926 20:40:20.717965 31876 raft_consensus.cc:348] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } }
I20260926 20:40:20.718456 31876 raft_consensus.cc:374] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:20.718539 31878 raft_consensus.cc:374] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:20.718617 31876 raft_consensus.cc:729] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c305b130115a46fd8a0dd27815c7dbbc, State: Initialized, Role: FOLLOWER
I20260926 20:40:20.718715 31878 raft_consensus.cc:729] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69a1f31efdcc44e7a1dd36ea8e70c6d7, State: Initialized, Role: FOLLOWER
I20260926 20:40:20.718925 31877 ts_tablet_manager.cc:1428] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260926 20:40:20.719533 31876 consensus_queue.cc:260] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } }
I20260926 20:40:20.719545 31878 consensus_queue.cc:260] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } }
I20260926 20:40:20.721886 31876 ts_tablet_manager.cc:1428] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc: Time spent starting tablet: real 0.016s user 0.012s sys 0.003s
I20260926 20:40:20.722947 31878 ts_tablet_manager.cc:1428] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
W20260926 20:40:20.813730 31573 tablet.cc:2369] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:40:20.853634 31712 tablet.cc:2369] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:40:20.922739 31852 tablet.cc:2369] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:40:20.961687 31884 raft_consensus.cc:482] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:40:20.961974 31884 raft_consensus.cc:504] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } }
I20260926 20:40:20.963594 31884 leader_election.cc:288] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185), c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194:45199)
I20260926 20:40:20.972056 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:20.965391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:57849 (local address 127.23.44.194:45199)
0926 20:40:20.965632 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:20.965640 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:20.965845 (+ 205us) server_negotiation.cc:407] Connection header received
0926 20:40:20.966032 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:20.966039 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:20.966139 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:20.966266 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:20.967110 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.967978 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.970263 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.971024 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.971239 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:20.971320 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:20.971644 (+ 324us) server_negotiation.cc:299] Negotiation successful
0926 20:40:20.971776 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260926 20:40:20.972188 31886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:20.965004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:45199 (local address 127.23.44.193:57849)
0926 20:40:20.965723 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:20.965741 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:20.965882 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:20.966384 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:20.966389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:20.966420 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:20.966902 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:20.966911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.968136 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.968142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:20.970087 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:20.970100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.971265 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.971272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:20.971430 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:20.971479 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 20:40:20.971605 (+ 126us) client_negotiation.cc:240] Negotiation successful
0926 20:40:20.971766 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":30,"thread_start_us":373,"threads_started":1}
I20260926 20:40:20.972812 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:20.963943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:41185 (local address 127.23.44.193:42711)
0926 20:40:20.964953 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:20.964976 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:20.965272 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:20.966081 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:20.966087 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:20.966118 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:20.966736 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:20.966753 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.968040 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.968048 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:20.970137 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:20.970150 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.971827 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.971834 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:20.972032 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:20.972057 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 20:40:20.972179 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 20:40:20.972386 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"mutex_wait_us":99,"thread_start_us":532,"threads_started":1}
I20260926 20:40:20.972811 31871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:20.964078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:42711 (local address 127.23.44.195:41185)
0926 20:40:20.964613 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:20.964624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:20.965159 (+ 535us) server_negotiation.cc:407] Connection header received
0926 20:40:20.965539 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:20.965546 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:20.965664 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:20.965753 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:20.966912 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.967889 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.970333 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:20.971510 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:20.971768 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:20.971906 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:20.972202 (+ 296us) server_negotiation.cc:299] Negotiation successful
0926 20:40:20.972582 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
I20260926 20:40:20.973417 31661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90aaa3a1cdcf4458b6ff25507eb8cbe5" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c305b130115a46fd8a0dd27815c7dbbc" is_pre_election: true
I20260926 20:40:20.973922 31661 raft_consensus.cc:2454] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 0.
I20260926 20:40:20.973876 31801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90aaa3a1cdcf4458b6ff25507eb8cbe5" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea36b613dc84692831b4ea0079b8dd8" is_pre_election: true
I20260926 20:40:20.974298 31801 raft_consensus.cc:2454] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 0.
I20260926 20:40:20.974611 31452 leader_election.cc:302] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69a1f31efdcc44e7a1dd36ea8e70c6d7, c305b130115a46fd8a0dd27815c7dbbc; no voters:
I20260926 20:40:20.974995 31884 raft_consensus.cc:2781] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:40:20.975168 31884 raft_consensus.cc:482] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:40:20.975279 31884 raft_consensus.cc:3037] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:20.977579 31884 raft_consensus.cc:504] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } }
I20260926 20:40:20.978204 31884 leader_election.cc:288] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 election: Requested vote from peers fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185), c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194:45199)
I20260926 20:40:20.978734 31801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90aaa3a1cdcf4458b6ff25507eb8cbe5" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea36b613dc84692831b4ea0079b8dd8"
I20260926 20:40:20.978847 31661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90aaa3a1cdcf4458b6ff25507eb8cbe5" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c305b130115a46fd8a0dd27815c7dbbc"
I20260926 20:40:20.978957 31801 raft_consensus.cc:3037] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:20.979096 31661 raft_consensus.cc:3037] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:20.981495 31661 raft_consensus.cc:2454] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 1.
I20260926 20:40:20.981539 31801 raft_consensus.cc:2454] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 1.
I20260926 20:40:20.981942 31452 leader_election.cc:302] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69a1f31efdcc44e7a1dd36ea8e70c6d7, c305b130115a46fd8a0dd27815c7dbbc; no voters:
I20260926 20:40:20.982322 31884 raft_consensus.cc:2781] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:20.982930 31884 raft_consensus.cc:686] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 LEADER]: Becoming Leader. State: Replica: 69a1f31efdcc44e7a1dd36ea8e70c6d7, State: Running, Role: LEADER
I20260926 20:40:20.983474 31884 consensus_queue.cc:237] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } }
I20260926 20:40:20.989262 31365 catalog_manager.cc:5495] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69a1f31efdcc44e7a1dd36ea8e70c6d7 (127.23.44.193). New cstate: current_term: 1 leader_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } health_report { overall_health: UNKNOWN } } }
I20260926 20:40:21.043623 31873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.038394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49030 (local address 127.23.44.193:33983)
0926 20:40:21.038633 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:21.038639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:21.038660 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:40:21.038789 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:21.038792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:21.038889 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:21.039049 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:40:21.039418 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.040115 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.041461 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.041886 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.042020 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.042464 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:40:21.043001 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:40:21.043164 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:21.043259 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 20:40:21.043359 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260926 20:40:21.055871 31661 raft_consensus.cc:1261] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Refusing update from remote peer 69a1f31efdcc44e7a1dd36ea8e70c6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:21.056268 31801 raft_consensus.cc:1261] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Refusing update from remote peer 69a1f31efdcc44e7a1dd36ea8e70c6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:21.057449 31884 consensus_queue.cc:1035] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:21.057922 31887 consensus_queue.cc:1035] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:21.077246 31892 mvcc.cc:205] Tried to move back new op lower bound from 7333704585428008960 to 7333704585153019904. Current Snapshot: MvccSnapshot[applied={T|T < 7333704585428008960}]
I20260926 20:40:21.092875 31365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:21.093163 31365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:21.096880 31365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c: Sending DeleteTablet for 3 replicas of tablet 90aaa3a1cdcf4458b6ff25507eb8cbe5
I20260926 20:40:21.098125 31781 tablet_service.cc:1514] Processing DeleteTablet for tablet 90aaa3a1cdcf4458b6ff25507eb8cbe5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:21 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:43712
I20260926 20:40:21.098346 31502 tablet_service.cc:1514] Processing DeleteTablet for tablet 90aaa3a1cdcf4458b6ff25507eb8cbe5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:21 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:49024
I20260926 20:40:21.099663 31641 tablet_service.cc:1514] Processing DeleteTablet for tablet 90aaa3a1cdcf4458b6ff25507eb8cbe5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:21 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:36190
I20260926 20:40:21.105409 31365 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:40:21.106961 31365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:40:21.110782 31900 tablet_replica.cc:330] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8: stopping tablet replica
I20260926 20:40:21.111946 31902 tablet_replica.cc:330] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc: stopping tablet replica
I20260926 20:40:21.112205 31901 tablet_replica.cc:330] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: stopping tablet replica
I20260926 20:40:21.112453 31902 raft_consensus.cc:2229] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:21.112941 31901 raft_consensus.cc:2229] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:40:21.113365 31902 raft_consensus.cc:2258] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:21.111449 31900 raft_consensus.cc:2229] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:21.113677 31901 raft_consensus.cc:2258] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:21.114120 31900 raft_consensus.cc:2258] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:21.116497 31902 ts_tablet_manager.cc:1905] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:21.116504 31901 ts_tablet_manager.cc:1905] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:21.119619 31641 tablet_service.cc:1467] Processing CreateTablet for tablet e30c4ec8b1384f968fbdeb8fa4abd302 (DEFAULT_TABLE table=test-table [id=4f065811e9f24df5a516bd633a053552]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:21.120234 31641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e30c4ec8b1384f968fbdeb8fa4abd302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:21.123916 31901 ts_tablet_manager.cc:1918] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:21.125602 31900 ts_tablet_manager.cc:1905] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:21.128104 31876 tablet_bootstrap.cc:492] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc: Bootstrap starting.
I20260926 20:40:21.129873 31876 tablet_bootstrap.cc:654] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:21.129890 31781 tablet_service.cc:1467] Processing CreateTablet for tablet e30c4ec8b1384f968fbdeb8fa4abd302 (DEFAULT_TABLE table=test-table [id=4f065811e9f24df5a516bd633a053552]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:21.130645 31781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e30c4ec8b1384f968fbdeb8fa4abd302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:21.133484 31900 ts_tablet_manager.cc:1918] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:21.129140 31502 tablet_service.cc:1467] Processing CreateTablet for tablet e30c4ec8b1384f968fbdeb8fa4abd302 (DEFAULT_TABLE table=test-table [id=4f065811e9f24df5a516bd633a053552]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:21.133884 31901 log.cc:1196] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/90aaa3a1cdcf4458b6ff25507eb8cbe5
I20260926 20:40:21.134344 31502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e30c4ec8b1384f968fbdeb8fa4abd302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:21.134721 31900 log.cc:1196] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/90aaa3a1cdcf4458b6ff25507eb8cbe5
I20260926 20:40:21.135634 31901 ts_tablet_manager.cc:1939] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Deleting consensus metadata
I20260926 20:40:21.136385 31900 ts_tablet_manager.cc:1939] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P fea36b613dc84692831b4ea0079b8dd8: Deleting consensus metadata
I20260926 20:40:21.136865 31902 ts_tablet_manager.cc:1918] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:21.137253 31902 log.cc:1196] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/90aaa3a1cdcf4458b6ff25507eb8cbe5
I20260926 20:40:21.138466 31902 ts_tablet_manager.cc:1939] T 90aaa3a1cdcf4458b6ff25507eb8cbe5 P c305b130115a46fd8a0dd27815c7dbbc: Deleting consensus metadata
I20260926 20:40:21.140400 31349 catalog_manager.cc:4841] TS 69a1f31efdcc44e7a1dd36ea8e70c6d7 (127.23.44.193:33983): tablet 90aaa3a1cdcf4458b6ff25507eb8cbe5 (table test-table [id=19927618ed0048dda15547bf41ff3d0e]) successfully deleted
I20260926 20:40:21.140573 31877 tablet_bootstrap.cc:492] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8: Bootstrap starting.
I20260926 20:40:21.141592 31349 catalog_manager.cc:4841] TS fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185): tablet 90aaa3a1cdcf4458b6ff25507eb8cbe5 (table test-table [id=19927618ed0048dda15547bf41ff3d0e]) successfully deleted
I20260926 20:40:21.142114 31878 tablet_bootstrap.cc:492] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Bootstrap starting.
I20260926 20:40:21.142262 31877 tablet_bootstrap.cc:654] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:21.143795 31349 catalog_manager.cc:4841] TS c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194:45199): tablet 90aaa3a1cdcf4458b6ff25507eb8cbe5 (table test-table [id=19927618ed0048dda15547bf41ff3d0e]) successfully deleted
I20260926 20:40:21.144635 31878 tablet_bootstrap.cc:654] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:21.146698 31876 tablet_bootstrap.cc:492] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc: No bootstrap required, opened a new log
I20260926 20:40:21.148676 31876 ts_tablet_manager.cc:1397] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.004s
I20260926 20:40:21.149824 31876 raft_consensus.cc:348] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.150349 31876 raft_consensus.cc:374] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:21.150530 31876 raft_consensus.cc:729] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c305b130115a46fd8a0dd27815c7dbbc, State: Initialized, Role: FOLLOWER
I20260926 20:40:21.151014 31876 consensus_queue.cc:260] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.154808 31878 tablet_bootstrap.cc:492] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: No bootstrap required, opened a new log
I20260926 20:40:21.155074 31878 ts_tablet_manager.cc:1397] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260926 20:40:21.156142 31878 raft_consensus.cc:348] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.156492 31878 raft_consensus.cc:374] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:21.156724 31878 raft_consensus.cc:729] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69a1f31efdcc44e7a1dd36ea8e70c6d7, State: Initialized, Role: FOLLOWER
I20260926 20:40:21.156745 31876 ts_tablet_manager.cc:1428] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260926 20:40:21.157145 31878 consensus_queue.cc:260] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.157925 31878 ts_tablet_manager.cc:1428] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 20:40:21.162071 31877 tablet_bootstrap.cc:492] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8: No bootstrap required, opened a new log
I20260926 20:40:21.162338 31877 ts_tablet_manager.cc:1397] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.017s
I20260926 20:40:21.163266 31877 raft_consensus.cc:348] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.163532 31877 raft_consensus.cc:374] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:21.163631 31877 raft_consensus.cc:729] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea36b613dc84692831b4ea0079b8dd8, State: Initialized, Role: FOLLOWER
I20260926 20:40:21.163975 31877 consensus_queue.cc:260] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.164745 31877 ts_tablet_manager.cc:1428] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 20:40:21.171530 31897 raft_consensus.cc:482] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:40:21.172073 31897 raft_consensus.cc:504] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.173033 31897 leader_election.cc:288] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194:45199), fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185)
I20260926 20:40:21.173609 31661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30c4ec8b1384f968fbdeb8fa4abd302" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c305b130115a46fd8a0dd27815c7dbbc" is_pre_election: true
I20260926 20:40:21.173759 31801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30c4ec8b1384f968fbdeb8fa4abd302" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea36b613dc84692831b4ea0079b8dd8" is_pre_election: true
I20260926 20:40:21.173866 31661 raft_consensus.cc:2454] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 0.
I20260926 20:40:21.174026 31801 raft_consensus.cc:2454] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 0.
I20260926 20:40:21.174379 31452 leader_election.cc:302] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69a1f31efdcc44e7a1dd36ea8e70c6d7, c305b130115a46fd8a0dd27815c7dbbc; no voters:
I20260926 20:40:21.174669 31897 raft_consensus.cc:2781] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:40:21.174813 31897 raft_consensus.cc:482] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:40:21.174922 31897 raft_consensus.cc:3037] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:21.178651 31897 raft_consensus.cc:504] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.179463 31897 leader_election.cc:288] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 election: Requested vote from peers c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194:45199), fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185)
I20260926 20:40:21.180409 31661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30c4ec8b1384f968fbdeb8fa4abd302" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c305b130115a46fd8a0dd27815c7dbbc"
I20260926 20:40:21.180521 31801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30c4ec8b1384f968fbdeb8fa4abd302" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea36b613dc84692831b4ea0079b8dd8"
I20260926 20:40:21.180785 31661 raft_consensus.cc:3037] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:21.180805 31801 raft_consensus.cc:3037] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:21.183777 31661 raft_consensus.cc:2454] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 1.
I20260926 20:40:21.184170 31801 raft_consensus.cc:2454] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 1.
I20260926 20:40:21.184350 31452 leader_election.cc:302] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69a1f31efdcc44e7a1dd36ea8e70c6d7, c305b130115a46fd8a0dd27815c7dbbc; no voters:
I20260926 20:40:21.184772 31897 raft_consensus.cc:2781] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:21.185040 31897 raft_consensus.cc:686] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 LEADER]: Becoming Leader. State: Replica: 69a1f31efdcc44e7a1dd36ea8e70c6d7, State: Running, Role: LEADER
I20260926 20:40:21.185389 31897 consensus_queue.cc:237] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.188592 31365 catalog_manager.cc:5495] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69a1f31efdcc44e7a1dd36ea8e70c6d7 (127.23.44.193). New cstate: current_term: 1 leader_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } health_report { overall_health: UNKNOWN } } }
Sep 26 20:40:21 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455217, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.254@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
I20260926 20:40:21.263526 31912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.247502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47442 (local address 127.23.44.254:39411)
0926 20:40:21.248224 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:21.248236 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:21.248464 (+ 228us) server_negotiation.cc:407] Connection header received
0926 20:40:21.248747 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:21.248753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:21.248826 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:21.248944 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:21.250065 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.250826 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.252063 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.252487 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.252694 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.256418 (+ 3724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:21.256424 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:21.256427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:21.256457 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:21.259446 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:21.260355 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:21.260364 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:21.260370 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:21.260509 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:21.261144 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:21.261153 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:21.261160 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:21.261734 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:21.262162 (+ 428us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:21.262708 (+ 546us) server_negotiation.cc:299] Negotiation successful
0926 20:40:21.263171 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":242,"server-negotiator.queue_time_us":540,"thread_start_us":357,"threads_started":1}
I20260926 20:40:21.263533 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.247493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39411 (local address 127.0.0.1:47442)
0926 20:40:21.248357 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:21.248372 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:21.248608 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:21.248969 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:21.248973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:21.249695 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:21.249888 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:21.249897 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.250970 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.250976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:21.251908 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:21.251918 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.252715 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.252721 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:21.252899 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.253632 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:21.253653 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:21.256188 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:21.259706 (+ 3518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:21.259725 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:21.259734 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:21.260217 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:21.260696 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:21.260702 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:21.260706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:21.260854 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:21.262120 (+ 1266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:21.262132 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:21.262396 (+ 264us) client_negotiation.cc:769] Sending connection context
0926 20:40:21.262586 (+ 190us) client_negotiation.cc:240] Negotiation successful
0926 20:40:21.263042 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":1,"thread_start_us":591,"threads_started":1}
I20260926 20:40:21.269556 31365 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:47442:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 20:40:21.270635 31365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:40:21.278607 31502 tablet_service.cc:1467] Processing CreateTablet for tablet 83af561466d04c1b9b464b004dec6472 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1a69b65595ea4b39aa9c0ad3f909076b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:40:21.279336 31502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83af561466d04c1b9b464b004dec6472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:21.279264 31641 tablet_service.cc:1467] Processing CreateTablet for tablet 83af561466d04c1b9b464b004dec6472 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1a69b65595ea4b39aa9c0ad3f909076b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:40:21.279325 31781 tablet_service.cc:1467] Processing CreateTablet for tablet 83af561466d04c1b9b464b004dec6472 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1a69b65595ea4b39aa9c0ad3f909076b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 20:40:21.279806 31641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83af561466d04c1b9b464b004dec6472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:21.279870 31781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83af561466d04c1b9b464b004dec6472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:21.287981 31877 tablet_bootstrap.cc:492] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8: Bootstrap starting.
I20260926 20:40:21.288422 31878 tablet_bootstrap.cc:492] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Bootstrap starting.
I20260926 20:40:21.289944 31877 tablet_bootstrap.cc:654] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:21.290479 31878 tablet_bootstrap.cc:654] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:21.291755 31876 tablet_bootstrap.cc:492] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc: Bootstrap starting.
I20260926 20:40:21.293514 31876 tablet_bootstrap.cc:654] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:21.293919 31877 tablet_bootstrap.cc:492] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8: No bootstrap required, opened a new log
I20260926 20:40:21.294099 31877 ts_tablet_manager.cc:1397] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20260926 20:40:21.294920 31877 raft_consensus.cc:348] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.295212 31877 raft_consensus.cc:374] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:21.295326 31877 raft_consensus.cc:729] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea36b613dc84692831b4ea0079b8dd8, State: Initialized, Role: FOLLOWER
I20260926 20:40:21.295667 31877 consensus_queue.cc:260] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.296204 31882 tablet_replica.cc:439] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } }
I20260926 20:40:21.297513 31882 tablet_replica.cc:442] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:40:21.298099 31877 ts_tablet_manager.cc:1428] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260926 20:40:21.300441 31878 tablet_bootstrap.cc:492] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: No bootstrap required, opened a new log
I20260926 20:40:21.300758 31878 ts_tablet_manager.cc:1397] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.007s
I20260926 20:40:21.301764 31878 raft_consensus.cc:348] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.302027 31878 raft_consensus.cc:374] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:21.302124 31878 raft_consensus.cc:729] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69a1f31efdcc44e7a1dd36ea8e70c6d7, State: Initialized, Role: FOLLOWER
I20260926 20:40:21.302394 31878 consensus_queue.cc:260] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.302867 31897 tablet_replica.cc:439] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } }
I20260926 20:40:21.303212 31897 tablet_replica.cc:442] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:40:21.303288 31878 ts_tablet_manager.cc:1428] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 20:40:21.305969 31876 tablet_bootstrap.cc:492] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc: No bootstrap required, opened a new log
I20260926 20:40:21.306185 31876 ts_tablet_manager.cc:1397] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.009s
I20260926 20:40:21.307040 31876 raft_consensus.cc:348] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.307349 31876 raft_consensus.cc:374] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:21.307444 31876 raft_consensus.cc:729] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c305b130115a46fd8a0dd27815c7dbbc, State: Initialized, Role: FOLLOWER
I20260926 20:40:21.307794 31876 consensus_queue.cc:260] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.308257 31883 tablet_replica.cc:439] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } }
I20260926 20:40:21.308593 31883 tablet_replica.cc:442] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:40:21.309783 31876 ts_tablet_manager.cc:1428] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260926 20:40:21.325166 31883 raft_consensus.cc:482] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:40:21.325418 31883 raft_consensus.cc:504] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.327476 31883 leader_election.cc:288] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69a1f31efdcc44e7a1dd36ea8e70c6d7 (127.23.44.193:33983), fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185)
I20260926 20:40:21.339850 31873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.328025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:32923 (local address 127.23.44.193:33983)
0926 20:40:21.328283 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:21.328295 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:21.330184 (+ 1889us) server_negotiation.cc:407] Connection header received
0926 20:40:21.330502 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:21.330507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:21.330581 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:21.330643 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:21.331527 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.332748 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.335407 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.336820 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.336988 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.337094 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:21.339327 (+ 2233us) server_negotiation.cc:299] Negotiation successful
0926 20:40:21.339491 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 20:40:21.340975 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.327914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:33983 (local address 127.23.44.194:32923)
0926 20:40:21.330063 (+ 2149us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:21.330084 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:21.330338 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:21.330920 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:21.330924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:21.330951 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:21.331344 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:21.331356 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.332941 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.332947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:21.335212 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:21.335228 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.338882 (+ 3654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.338888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:21.339124 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.339162 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 20:40:21.340364 (+ 1202us) client_negotiation.cc:240] Negotiation successful
0926 20:40:21.340528 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1996,"thread_start_us":1842,"threads_started":1}
I20260926 20:40:21.342146 31527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83af561466d04c1b9b464b004dec6472" candidate_uuid: "c305b130115a46fd8a0dd27815c7dbbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" is_pre_election: true
I20260926 20:40:21.342629 31527 raft_consensus.cc:2454] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c305b130115a46fd8a0dd27815c7dbbc in term 0.
I20260926 20:40:21.343438 31592 leader_election.cc:302] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69a1f31efdcc44e7a1dd36ea8e70c6d7, c305b130115a46fd8a0dd27815c7dbbc; no voters:
I20260926 20:40:21.343909 31883 raft_consensus.cc:2781] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:40:21.344043 31883 raft_consensus.cc:482] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:40:21.344130 31883 raft_consensus.cc:3037] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:21.347463 31883 raft_consensus.cc:504] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.348232 31871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.330650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:49213 (local address 127.23.44.195:41185)
0926 20:40:21.330923 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:21.330934 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:21.341389 (+ 10455us) server_negotiation.cc:407] Connection header received
0926 20:40:21.341865 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:21.341872 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:21.341972 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:21.342066 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:21.342853 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.344037 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.346204 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.347108 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.347340 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.347428 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:21.347764 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 20:40:21.347908 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260926 20:40:21.348337 31918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.330330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:41185 (local address 127.23.44.194:49213)
0926 20:40:21.341252 (+ 10922us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:21.341281 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:21.341636 (+ 355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:21.342274 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:21.342280 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:21.342309 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:21.342661 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:21.342673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.344203 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.344212 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:21.346001 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:21.346014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.347376 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.347383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:21.347587 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.347614 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 20:40:21.347737 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 20:40:21.347890 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10755,"thread_start_us":10340,"threads_started":1}
I20260926 20:40:21.349160 31801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83af561466d04c1b9b464b004dec6472" candidate_uuid: "c305b130115a46fd8a0dd27815c7dbbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea36b613dc84692831b4ea0079b8dd8" is_pre_election: true
I20260926 20:40:21.349304 31883 leader_election.cc:288] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [CANDIDATE]: Term 1 election: Requested vote from peers 69a1f31efdcc44e7a1dd36ea8e70c6d7 (127.23.44.193:33983), fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185)
I20260926 20:40:21.349455 31801 raft_consensus.cc:2454] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c305b130115a46fd8a0dd27815c7dbbc in term 0.
I20260926 20:40:21.350065 31527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83af561466d04c1b9b464b004dec6472" candidate_uuid: "c305b130115a46fd8a0dd27815c7dbbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7"
I20260926 20:40:21.350284 31527 raft_consensus.cc:3037] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:21.350452 31801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83af561466d04c1b9b464b004dec6472" candidate_uuid: "c305b130115a46fd8a0dd27815c7dbbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea36b613dc84692831b4ea0079b8dd8"
I20260926 20:40:21.350670 31801 raft_consensus.cc:3037] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:21.352864 31801 raft_consensus.cc:2454] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c305b130115a46fd8a0dd27815c7dbbc in term 1.
I20260926 20:40:21.353333 31592 leader_election.cc:302] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c305b130115a46fd8a0dd27815c7dbbc, fea36b613dc84692831b4ea0079b8dd8; no voters:
I20260926 20:40:21.353680 31883 raft_consensus.cc:2781] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:21.353745 31527 raft_consensus.cc:2454] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c305b130115a46fd8a0dd27815c7dbbc in term 1.
I20260926 20:40:21.354206 31883 raft_consensus.cc:686] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [term 1 LEADER]: Becoming Leader. State: Replica: c305b130115a46fd8a0dd27815c7dbbc, State: Running, Role: LEADER
I20260926 20:40:21.354692 31883 consensus_queue.cc:237] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.357218 31919 tablet_replica.cc:439] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc: TxnStatusTablet state changed. Reason: New leader c305b130115a46fd8a0dd27815c7dbbc. Latest consensus state: current_term: 1 leader_uuid: "c305b130115a46fd8a0dd27815c7dbbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } }
I20260926 20:40:21.357651 31919 tablet_replica.cc:442] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:40:21.358613 31921 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 20:40:21.358841 31921 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 20:40:21.359059 31365 catalog_manager.cc:5495] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc reported cstate change: term changed from 0 to 1, leader changed from <none> to c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194). New cstate: current_term: 1 leader_uuid: "c305b130115a46fd8a0dd27815c7dbbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } health_report { overall_health: UNKNOWN } } }
I20260926 20:40:21.396106 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.387756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:45199 (local address 127.0.0.1:36194)
0926 20:40:21.388035 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:21.388059 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:21.388214 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:21.388779 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:21.388784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:21.388796 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:40:21.388991 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:21.389002 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.390888 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.390894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:21.392621 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:21.392637 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.393668 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.393677 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:21.393935 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.394017 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:40:21.395446 (+ 1429us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:40:21.395469 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 20:40:21.395566 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 20:40:21.395739 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 20:40:21.396149 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.388136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36194 (local address 127.23.44.194:45199)
0926 20:40:21.388300 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:21.388306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:21.388326 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:40:21.388427 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:21.388433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:21.388582 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:21.388725 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:40:21.389315 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.390688 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.392815 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.393471 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.393618 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.394236 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:40:21.395049 (+ 813us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:40:21.395505 (+ 456us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:21.395589 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 20:40:21.395746 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 20:40:21.402055 31527 raft_consensus.cc:1261] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Refusing update from remote peer c305b130115a46fd8a0dd27815c7dbbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:21.403391 31919 consensus_queue.cc:1035] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:21.404526 31801 raft_consensus.cc:1261] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Refusing update from remote peer c305b130115a46fd8a0dd27815c7dbbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:21.406107 31919 consensus_queue.cc:1035] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:21.430159 31897 tablet_replica.cc:439] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: TxnStatusTablet state changed. Reason: New leader c305b130115a46fd8a0dd27815c7dbbc. Latest consensus state: current_term: 1 leader_uuid: "c305b130115a46fd8a0dd27815c7dbbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } }
I20260926 20:40:21.430560 31897 tablet_replica.cc:442] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:40:21.434012 31919 tablet_replica.cc:439] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c305b130115a46fd8a0dd27815c7dbbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } }
I20260926 20:40:21.434283 31919 tablet_replica.cc:442] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:40:21.434471 31891 mvcc.cc:205] Tried to move back new op lower bound from 7333704586852065280 to 7333704586673041408. Current Snapshot: MvccSnapshot[applied={T|T < 7333704586852065280}]
I20260926 20:40:21.443066 31882 tablet_replica.cc:439] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8: TxnStatusTablet state changed. Reason: New leader c305b130115a46fd8a0dd27815c7dbbc. Latest consensus state: current_term: 1 leader_uuid: "c305b130115a46fd8a0dd27815c7dbbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } }
I20260926 20:40:21.443400 31882 tablet_replica.cc:442] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:40:21.453156 31926 tablet_replica.cc:439] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c305b130115a46fd8a0dd27815c7dbbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } }
I20260926 20:40:21.453545 31926 tablet_replica.cc:442] T 83af561466d04c1b9b464b004dec6472 P c305b130115a46fd8a0dd27815c7dbbc: This TxnStatusTablet replica's current role is: LEADER
I20260926 20:40:21.455384 31882 tablet_replica.cc:439] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c305b130115a46fd8a0dd27815c7dbbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } }
I20260926 20:40:21.455624 31882 tablet_replica.cc:442] T 83af561466d04c1b9b464b004dec6472 P fea36b613dc84692831b4ea0079b8dd8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:40:21.460901 31897 tablet_replica.cc:439] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c305b130115a46fd8a0dd27815c7dbbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } } }
I20260926 20:40:21.461229 31897 tablet_replica.cc:442] T 83af561466d04c1b9b464b004dec6472 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 20:40:21.486096 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.478805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:55763 (local address 127.23.44.194:45199)
0926 20:40:21.478989 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:21.478996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:21.479273 (+ 277us) server_negotiation.cc:407] Connection header received
0926 20:40:21.479511 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:21.479517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:21.479603 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:21.479703 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:40:21.480277 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.481571 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.483817 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.484570 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.484714 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.485270 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:40:21.485413 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:40:21.485508 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:21.485689 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 20:40:21.485774 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 20:40:21.486258 31929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.478450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:45199 (local address 127.23.44.193:55763)
0926 20:40:21.479143 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:21.479169 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:21.479338 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:21.479768 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:21.479773 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:21.479786 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:40:21.480088 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:21.480101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.481730 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.481736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:21.483640 (+ 1904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:21.483657 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.484773 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.484780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:21.484974 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.485047 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:40:21.485535 (+ 488us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:40:21.485554 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 20:40:21.485656 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 20:40:21.485813 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":386,"threads_started":1}
I20260926 20:40:21.500257 31801 raft_consensus.cc:1261] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Refusing update from remote peer 69a1f31efdcc44e7a1dd36ea8e70c6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:21.500280 31661 raft_consensus.cc:1261] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Refusing update from remote peer 69a1f31efdcc44e7a1dd36ea8e70c6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:21.501009 31884 consensus_queue.cc:1035] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:21.501276 31897 consensus_queue.cc:1035] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:21.518153 31890 mvcc.cc:205] Tried to move back new op lower bound from 7333704587258621952 to 7333704585978511360. Current Snapshot: MvccSnapshot[applied={T|T < 7333704587258621952}]
I20260926 20:40:21.582149 31873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.560400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:34743 (local address 127.23.44.193:33983)
0926 20:40:21.560833 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:21.560843 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:21.574177 (+ 13334us) server_negotiation.cc:407] Connection header received
0926 20:40:21.574494 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:21.574505 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:21.574599 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:21.574704 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:40:21.575713 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.577000 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.579351 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.580178 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.580350 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.581117 (+ 767us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:40:21.581265 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:40:21.581334 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:21.581689 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 20:40:21.581791 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290}
I20260926 20:40:21.583099 31933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.556423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:33983 (local address 127.23.44.194:34743)
0926 20:40:21.574058 (+ 17635us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:21.574085 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:21.574310 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:21.575034 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:21.575040 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:21.575053 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 20:40:21.575497 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:21.575509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.577239 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.577244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:21.579153 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:21.579171 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.580544 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.580584 (+ 40us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:21.580789 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.580883 (+ 94us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 20:40:21.581480 (+ 597us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 20:40:21.581494 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 20:40:21.582367 (+ 873us) client_negotiation.cc:240] Negotiation successful
0926 20:40:21.582627 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17406,"thread_start_us":17757,"threads_started":1}
I20260926 20:40:21.638082 31364 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:21.638403 31364 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:21.642589 31364 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c: Sending DeleteTablet for 3 replicas of tablet e30c4ec8b1384f968fbdeb8fa4abd302
I20260926 20:40:21.644013 31781 tablet_service.cc:1514] Processing DeleteTablet for tablet e30c4ec8b1384f968fbdeb8fa4abd302 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:21 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:43712
I20260926 20:40:21.644312 31900 tablet_replica.cc:330] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8: stopping tablet replica
I20260926 20:40:21.644273 31502 tablet_service.cc:1514] Processing DeleteTablet for tablet e30c4ec8b1384f968fbdeb8fa4abd302 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:21 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:49024
I20260926 20:40:21.644512 31900 raft_consensus.cc:2229] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:21.644735 31900 raft_consensus.cc:2258] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:21.645015 31901 tablet_replica.cc:330] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: stopping tablet replica
I20260926 20:40:21.645228 31901 raft_consensus.cc:2229] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:40:21.645548 31901 raft_consensus.cc:2258] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:21.646987 31900 ts_tablet_manager.cc:1905] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:21.652828 31641 tablet_service.cc:1514] Processing DeleteTablet for tablet e30c4ec8b1384f968fbdeb8fa4abd302 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:21 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:36190
I20260926 20:40:21.654551 31901 ts_tablet_manager.cc:1905] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:21.671782 31900 ts_tablet_manager.cc:1918] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:40:21.671958 31900 log.cc:1196] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/e30c4ec8b1384f968fbdeb8fa4abd302
I20260926 20:40:21.672358 31900 ts_tablet_manager.cc:1939] T e30c4ec8b1384f968fbdeb8fa4abd302 P fea36b613dc84692831b4ea0079b8dd8: Deleting consensus metadata
I20260926 20:40:21.673686 31935 tablet_replica.cc:330] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc: stopping tablet replica
I20260926 20:40:21.673882 31935 raft_consensus.cc:2229] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:21.673931 31349 catalog_manager.cc:4841] TS fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185): tablet e30c4ec8b1384f968fbdeb8fa4abd302 (table test-table [id=4f065811e9f24df5a516bd633a053552]) successfully deleted
I20260926 20:40:21.674127 31935 raft_consensus.cc:2258] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:21.675163 31935 ts_tablet_manager.cc:1905] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:21.675155 31901 ts_tablet_manager.cc:1918] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:40:21.675407 31901 log.cc:1196] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/e30c4ec8b1384f968fbdeb8fa4abd302
I20260926 20:40:21.675912 31901 ts_tablet_manager.cc:1939] T e30c4ec8b1384f968fbdeb8fa4abd302 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Deleting consensus metadata
I20260926 20:40:21.677150 31349 catalog_manager.cc:4841] TS 69a1f31efdcc44e7a1dd36ea8e70c6d7 (127.23.44.193:33983): tablet e30c4ec8b1384f968fbdeb8fa4abd302 (table test-table [id=4f065811e9f24df5a516bd633a053552]) successfully deleted
I20260926 20:40:21.685074 31935 ts_tablet_manager.cc:1918] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:40:21.685324 31935 log.cc:1196] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/e30c4ec8b1384f968fbdeb8fa4abd302
I20260926 20:40:21.685838 31935 ts_tablet_manager.cc:1939] T e30c4ec8b1384f968fbdeb8fa4abd302 P c305b130115a46fd8a0dd27815c7dbbc: Deleting consensus metadata
I20260926 20:40:21.687060 31349 catalog_manager.cc:4841] TS c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194:45199): tablet e30c4ec8b1384f968fbdeb8fa4abd302 (table test-table [id=4f065811e9f24df5a516bd633a053552]) successfully deleted
Sep 26 20:40:21 dist-test-slave-4d5t krb5kdc[31320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 20:40:21 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:21 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 20:40:21.744180 31912 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:21.730279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47448 (local address 127.23.44.254:39411)
0926 20:40:21.730492 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:21.730498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:21.731064 (+ 566us) server_negotiation.cc:407] Connection header received
0926 20:40:21.731387 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:21.731394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:21.731480 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:21.731603 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:21.733445 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.734586 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.736276 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.737063 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.737251 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.743906 (+ 6655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47448: BlockingRecv error: recv got EOF from 127.0.0.1:47448 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Sep 26 20:40:21 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM
I20260926 20:40:21.765654 31912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.747568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47462 (local address 127.23.44.254:39411)
0926 20:40:21.747753 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:21.747758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:21.749080 (+ 1322us) server_negotiation.cc:407] Connection header received
0926 20:40:21.749328 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:21.749334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:21.749414 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:21.749546 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:21.750998 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.751800 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.753290 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.753788 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.753935 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.759285 (+ 5350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:21.759291 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:21.759295 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:21.759338 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:21.762559 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:21.763437 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:21.763448 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:21.763451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:21.763595 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:21.764048 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:21.764060 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:21.764063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:21.764358 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:21.764708 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:21.765006 (+ 298us) server_negotiation.cc:299] Negotiation successful
0926 20:40:21.765269 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 20:40:21.770289 31364 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:40:21.771039 31364 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:40:21.779366 31502 tablet_service.cc:1467] Processing CreateTablet for tablet 2326501b8ee24a79a9c0c44f83183045 (DEFAULT_TABLE table=test-table [id=a7bdb7723ad54410a74c7a82b6192f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:21.779851 31502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2326501b8ee24a79a9c0c44f83183045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:21.779851 31781 tablet_service.cc:1467] Processing CreateTablet for tablet 2326501b8ee24a79a9c0c44f83183045 (DEFAULT_TABLE table=test-table [id=a7bdb7723ad54410a74c7a82b6192f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:21.780481 31781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2326501b8ee24a79a9c0c44f83183045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:21.783496 31641 tablet_service.cc:1467] Processing CreateTablet for tablet 2326501b8ee24a79a9c0c44f83183045 (DEFAULT_TABLE table=test-table [id=a7bdb7723ad54410a74c7a82b6192f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:21.783923 31641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2326501b8ee24a79a9c0c44f83183045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:21.786218 31877 tablet_bootstrap.cc:492] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8: Bootstrap starting.
I20260926 20:40:21.787499 31877 tablet_bootstrap.cc:654] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:21.789548 31877 tablet_bootstrap.cc:492] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8: No bootstrap required, opened a new log
I20260926 20:40:21.789700 31877 ts_tablet_manager.cc:1397] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260926 20:40:21.790506 31877 raft_consensus.cc:348] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.790786 31877 raft_consensus.cc:374] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:21.790871 31877 raft_consensus.cc:729] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea36b613dc84692831b4ea0079b8dd8, State: Initialized, Role: FOLLOWER
I20260926 20:40:21.791098 31877 consensus_queue.cc:260] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.792725 31877 ts_tablet_manager.cc:1428] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260926 20:40:21.793058 31878 tablet_bootstrap.cc:492] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Bootstrap starting.
I20260926 20:40:21.795317 31878 tablet_bootstrap.cc:654] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:21.798533 31876 tablet_bootstrap.cc:492] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc: Bootstrap starting.
I20260926 20:40:21.801229 31876 tablet_bootstrap.cc:654] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:21.805079 31876 tablet_bootstrap.cc:492] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc: No bootstrap required, opened a new log
I20260926 20:40:21.805367 31876 ts_tablet_manager.cc:1397] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260926 20:40:21.806313 31876 raft_consensus.cc:348] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.806659 31876 raft_consensus.cc:374] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:21.806807 31876 raft_consensus.cc:729] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c305b130115a46fd8a0dd27815c7dbbc, State: Initialized, Role: FOLLOWER
I20260926 20:40:21.807121 31878 tablet_bootstrap.cc:492] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: No bootstrap required, opened a new log
I20260926 20:40:21.807283 31878 ts_tablet_manager.cc:1397] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.010s
I20260926 20:40:21.807166 31876 consensus_queue.cc:260] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.808101 31878 raft_consensus.cc:348] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.808414 31878 raft_consensus.cc:374] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:21.808435 31876 ts_tablet_manager.cc:1428] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 20:40:21.808624 31878 raft_consensus.cc:729] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69a1f31efdcc44e7a1dd36ea8e70c6d7, State: Initialized, Role: FOLLOWER
I20260926 20:40:21.808945 31878 consensus_queue.cc:260] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.810701 31878 ts_tablet_manager.cc:1428] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 20:40:21.895754 31897 raft_consensus.cc:482] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:40:21.895992 31897 raft_consensus.cc:504] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.896813 31897 leader_election.cc:288] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194:45199), fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185)
I20260926 20:40:21.897581 31800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2326501b8ee24a79a9c0c44f83183045" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea36b613dc84692831b4ea0079b8dd8" is_pre_election: true
I20260926 20:40:21.897584 31661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2326501b8ee24a79a9c0c44f83183045" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c305b130115a46fd8a0dd27815c7dbbc" is_pre_election: true
I20260926 20:40:21.897841 31661 raft_consensus.cc:2454] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 0.
I20260926 20:40:21.897836 31800 raft_consensus.cc:2454] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 0.
I20260926 20:40:21.898306 31452 leader_election.cc:302] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69a1f31efdcc44e7a1dd36ea8e70c6d7, c305b130115a46fd8a0dd27815c7dbbc; no voters:
I20260926 20:40:21.898581 31897 raft_consensus.cc:2781] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:40:21.898711 31897 raft_consensus.cc:482] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:40:21.898802 31897 raft_consensus.cc:3037] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:21.902072 31897 raft_consensus.cc:504] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.902776 31897 leader_election.cc:288] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 election: Requested vote from peers c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194:45199), fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185)
I20260926 20:40:21.903335 31800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2326501b8ee24a79a9c0c44f83183045" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea36b613dc84692831b4ea0079b8dd8"
I20260926 20:40:21.903345 31661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2326501b8ee24a79a9c0c44f83183045" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c305b130115a46fd8a0dd27815c7dbbc"
I20260926 20:40:21.903551 31800 raft_consensus.cc:3037] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:21.903589 31661 raft_consensus.cc:3037] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:21.905786 31800 raft_consensus.cc:2454] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 1.
I20260926 20:40:21.906239 31452 leader_election.cc:302] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69a1f31efdcc44e7a1dd36ea8e70c6d7, fea36b613dc84692831b4ea0079b8dd8; no voters:
I20260926 20:40:21.906562 31897 raft_consensus.cc:2781] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:21.906721 31897 raft_consensus.cc:686] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 LEADER]: Becoming Leader. State: Replica: 69a1f31efdcc44e7a1dd36ea8e70c6d7, State: Running, Role: LEADER
I20260926 20:40:21.906960 31661 raft_consensus.cc:2454] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 1.
I20260926 20:40:21.907029 31897 consensus_queue.cc:237] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:21.909989 31365 catalog_manager.cc:5495] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69a1f31efdcc44e7a1dd36ea8e70c6d7 (127.23.44.193). New cstate: current_term: 1 leader_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } health_report { overall_health: UNKNOWN } } }
I20260926 20:40:21.954144 31873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:21.948440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49040 (local address 127.23.44.193:33983)
0926 20:40:21.948755 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:21.948761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:21.948784 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:40:21.948971 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:21.948974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:21.949045 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:21.949133 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:40:21.949807 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.950605 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.952046 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:21.952459 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:21.952630 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:21.953296 (+ 666us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:40:21.953439 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:40:21.953528 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:21.953722 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 20:40:21.953790 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260926 20:40:21.957433 31800 raft_consensus.cc:1261] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Refusing update from remote peer 69a1f31efdcc44e7a1dd36ea8e70c6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:21.957458 31661 raft_consensus.cc:1261] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Refusing update from remote peer 69a1f31efdcc44e7a1dd36ea8e70c6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:21.958352 31897 consensus_queue.cc:1035] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:21.958729 31884 consensus_queue.cc:1035] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:21.976893 31365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:21.977152 31365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:21.980438 31365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c: Sending DeleteTablet for 3 replicas of tablet 2326501b8ee24a79a9c0c44f83183045
I20260926 20:40:21.981427 31641 tablet_service.cc:1514] Processing DeleteTablet for tablet 2326501b8ee24a79a9c0c44f83183045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:21 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:36190
I20260926 20:40:21.981567 31502 tablet_service.cc:1514] Processing DeleteTablet for tablet 2326501b8ee24a79a9c0c44f83183045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:21 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:49024
I20260926 20:40:21.981554 31781 tablet_service.cc:1514] Processing DeleteTablet for tablet 2326501b8ee24a79a9c0c44f83183045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:21 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:43712
I20260926 20:40:21.981738 31935 tablet_replica.cc:330] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc: stopping tablet replica
I20260926 20:40:21.981884 31900 tablet_replica.cc:330] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8: stopping tablet replica
I20260926 20:40:21.981935 31935 raft_consensus.cc:2229] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:21.982019 31900 raft_consensus.cc:2229] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:21.982163 31935 raft_consensus.cc:2258] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:21.982280 31901 tablet_replica.cc:330] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: stopping tablet replica
I20260926 20:40:21.982296 31900 raft_consensus.cc:2258] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:21.982445 31901 raft_consensus.cc:2229] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:40:21.982832 31901 raft_consensus.cc:2258] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:21.983248 31935 ts_tablet_manager.cc:1905] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:21.983011 31365 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:40:21.983777 31365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:40:21.983944 31901 ts_tablet_manager.cc:1905] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:21.985692 31900 ts_tablet_manager.cc:1905] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:21.991487 31900 ts_tablet_manager.cc:1918] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:21.991662 31900 log.cc:1196] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/2326501b8ee24a79a9c0c44f83183045
I20260926 20:40:21.992053 31900 ts_tablet_manager.cc:1939] T 2326501b8ee24a79a9c0c44f83183045 P fea36b613dc84692831b4ea0079b8dd8: Deleting consensus metadata
I20260926 20:40:21.992036 31901 ts_tablet_manager.cc:1918] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:21.992192 31901 log.cc:1196] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/2326501b8ee24a79a9c0c44f83183045
I20260926 20:40:21.993026 31901 ts_tablet_manager.cc:1939] T 2326501b8ee24a79a9c0c44f83183045 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Deleting consensus metadata
I20260926 20:40:21.993243 31349 catalog_manager.cc:4841] TS fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185): tablet 2326501b8ee24a79a9c0c44f83183045 (table test-table [id=a7bdb7723ad54410a74c7a82b6192f56]) successfully deleted
I20260926 20:40:21.993978 31935 ts_tablet_manager.cc:1918] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:21.994174 31935 log.cc:1196] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/2326501b8ee24a79a9c0c44f83183045
I20260926 20:40:21.994657 31935 ts_tablet_manager.cc:1939] T 2326501b8ee24a79a9c0c44f83183045 P c305b130115a46fd8a0dd27815c7dbbc: Deleting consensus metadata
I20260926 20:40:21.994988 31349 catalog_manager.cc:4841] TS 69a1f31efdcc44e7a1dd36ea8e70c6d7 (127.23.44.193:33983): tablet 2326501b8ee24a79a9c0c44f83183045 (table test-table [id=a7bdb7723ad54410a74c7a82b6192f56]) successfully deleted
I20260926 20:40:21.995196 31502 tablet_service.cc:1467] Processing CreateTablet for tablet 7348c157b7e04351ba985325f06451a2 (DEFAULT_TABLE table=test-table [id=8a13b1bd76434a89ac888170e77d957d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:21.995402 31641 tablet_service.cc:1467] Processing CreateTablet for tablet 7348c157b7e04351ba985325f06451a2 (DEFAULT_TABLE table=test-table [id=8a13b1bd76434a89ac888170e77d957d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:21.995659 31502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7348c157b7e04351ba985325f06451a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:21.995848 31641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7348c157b7e04351ba985325f06451a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:21.996193 31781 tablet_service.cc:1467] Processing CreateTablet for tablet 7348c157b7e04351ba985325f06451a2 (DEFAULT_TABLE table=test-table [id=8a13b1bd76434a89ac888170e77d957d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:21.996609 31781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7348c157b7e04351ba985325f06451a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:21.997504 31349 catalog_manager.cc:4841] TS c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194:45199): tablet 2326501b8ee24a79a9c0c44f83183045 (table test-table [id=a7bdb7723ad54410a74c7a82b6192f56]) successfully deleted
I20260926 20:40:22.001698 31876 tablet_bootstrap.cc:492] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc: Bootstrap starting.
I20260926 20:40:22.002763 31877 tablet_bootstrap.cc:492] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8: Bootstrap starting.
I20260926 20:40:22.002858 31878 tablet_bootstrap.cc:492] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Bootstrap starting.
I20260926 20:40:22.003414 31876 tablet_bootstrap.cc:654] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:22.004071 31878 tablet_bootstrap.cc:654] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:22.004644 31877 tablet_bootstrap.cc:654] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:22.006313 31878 tablet_bootstrap.cc:492] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: No bootstrap required, opened a new log
I20260926 20:40:22.006508 31878 ts_tablet_manager.cc:1397] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260926 20:40:22.007032 31876 tablet_bootstrap.cc:492] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc: No bootstrap required, opened a new log
I20260926 20:40:22.007273 31876 ts_tablet_manager.cc:1397] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20260926 20:40:22.007160 31878 raft_consensus.cc:348] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:22.007421 31878 raft_consensus.cc:374] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:22.007499 31878 raft_consensus.cc:729] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69a1f31efdcc44e7a1dd36ea8e70c6d7, State: Initialized, Role: FOLLOWER
I20260926 20:40:22.007751 31878 consensus_queue.cc:260] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:22.008191 31878 ts_tablet_manager.cc:1428] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260926 20:40:22.008360 31876 raft_consensus.cc:348] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:22.008816 31876 raft_consensus.cc:374] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:22.008929 31876 raft_consensus.cc:729] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c305b130115a46fd8a0dd27815c7dbbc, State: Initialized, Role: FOLLOWER
I20260926 20:40:22.009225 31876 consensus_queue.cc:260] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:22.009835 31877 tablet_bootstrap.cc:492] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8: No bootstrap required, opened a new log
I20260926 20:40:22.010066 31877 ts_tablet_manager.cc:1397] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.002s
I20260926 20:40:22.010558 31876 ts_tablet_manager.cc:1428] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 20:40:22.011324 31877 raft_consensus.cc:348] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:22.011757 31877 raft_consensus.cc:374] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:22.011862 31877 raft_consensus.cc:729] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea36b613dc84692831b4ea0079b8dd8, State: Initialized, Role: FOLLOWER
I20260926 20:40:22.012203 31877 consensus_queue.cc:260] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:22.012841 31877 ts_tablet_manager.cc:1428] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260926 20:40:22.080749 31884 raft_consensus.cc:482] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:40:22.080960 31884 raft_consensus.cc:504] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:22.081831 31884 leader_election.cc:288] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194:45199), fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185)
I20260926 20:40:22.082440 31661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7348c157b7e04351ba985325f06451a2" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c305b130115a46fd8a0dd27815c7dbbc" is_pre_election: true
I20260926 20:40:22.082470 31800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7348c157b7e04351ba985325f06451a2" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea36b613dc84692831b4ea0079b8dd8" is_pre_election: true
I20260926 20:40:22.082697 31661 raft_consensus.cc:2454] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 0.
I20260926 20:40:22.082754 31800 raft_consensus.cc:2454] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 0.
I20260926 20:40:22.083278 31452 leader_election.cc:302] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69a1f31efdcc44e7a1dd36ea8e70c6d7, c305b130115a46fd8a0dd27815c7dbbc; no voters:
I20260926 20:40:22.083616 31884 raft_consensus.cc:2781] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:40:22.083751 31884 raft_consensus.cc:482] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:40:22.083855 31884 raft_consensus.cc:3037] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:22.086081 31884 raft_consensus.cc:504] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:22.086671 31884 leader_election.cc:288] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 election: Requested vote from peers c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194:45199), fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185)
I20260926 20:40:22.087162 31661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7348c157b7e04351ba985325f06451a2" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c305b130115a46fd8a0dd27815c7dbbc"
I20260926 20:40:22.087360 31661 raft_consensus.cc:3037] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:22.087343 31800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7348c157b7e04351ba985325f06451a2" candidate_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea36b613dc84692831b4ea0079b8dd8"
I20260926 20:40:22.087606 31800 raft_consensus.cc:3037] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:22.089563 31661 raft_consensus.cc:2454] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 1.
I20260926 20:40:22.089912 31800 raft_consensus.cc:2454] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69a1f31efdcc44e7a1dd36ea8e70c6d7 in term 1.
I20260926 20:40:22.090070 31452 leader_election.cc:302] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69a1f31efdcc44e7a1dd36ea8e70c6d7, c305b130115a46fd8a0dd27815c7dbbc; no voters:
I20260926 20:40:22.090402 31884 raft_consensus.cc:2781] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:22.090576 31884 raft_consensus.cc:686] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 LEADER]: Becoming Leader. State: Replica: 69a1f31efdcc44e7a1dd36ea8e70c6d7, State: Running, Role: LEADER
I20260926 20:40:22.090873 31884 consensus_queue.cc:237] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } }
I20260926 20:40:22.094013 31364 catalog_manager.cc:5495] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69a1f31efdcc44e7a1dd36ea8e70c6d7 (127.23.44.193). New cstate: current_term: 1 leader_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69a1f31efdcc44e7a1dd36ea8e70c6d7" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 } health_report { overall_health: UNKNOWN } } }
I20260926 20:40:22.127861 31800 raft_consensus.cc:1261] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Refusing update from remote peer 69a1f31efdcc44e7a1dd36ea8e70c6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:22.128121 31661 raft_consensus.cc:1261] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Refusing update from remote peer 69a1f31efdcc44e7a1dd36ea8e70c6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:22.128768 31884 consensus_queue.cc:1035] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fea36b613dc84692831b4ea0079b8dd8" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:22.129086 31897 consensus_queue.cc:1035] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c305b130115a46fd8a0dd27815c7dbbc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:22.178926 31364 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:22.179209 31364 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:22.182610 31364 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c19d17dfa9e41269f5f69adce776c6c: Sending DeleteTablet for 3 replicas of tablet 7348c157b7e04351ba985325f06451a2
I20260926 20:40:22.183713 31641 tablet_service.cc:1514] Processing DeleteTablet for tablet 7348c157b7e04351ba985325f06451a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:22 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:36190
I20260926 20:40:22.183722 31502 tablet_service.cc:1514] Processing DeleteTablet for tablet 7348c157b7e04351ba985325f06451a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:22 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:49024
I20260926 20:40:22.184055 31901 tablet_replica.cc:330] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: stopping tablet replica
I20260926 20:40:22.184095 31781 tablet_service.cc:1514] Processing DeleteTablet for tablet 7348c157b7e04351ba985325f06451a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:22 UTC) from {username='oryx', principal='oryx/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:43712
I20260926 20:40:22.184295 31901 raft_consensus.cc:2229] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:40:22.184384 31900 tablet_replica.cc:330] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8: stopping tablet replica
I20260926 20:40:22.184406 31935 tablet_replica.cc:330] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc: stopping tablet replica
I20260926 20:40:22.184583 31900 raft_consensus.cc:2229] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:22.184640 31901 raft_consensus.cc:2258] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:22.184645 31935 raft_consensus.cc:2229] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:22.184777 31900 raft_consensus.cc:2258] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:22.184864 31935 raft_consensus.cc:2258] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:22.185829 31935 ts_tablet_manager.cc:1905] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:22.185846 31901 ts_tablet_manager.cc:1905] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:22.185853 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 31436
I20260926 20:40:22.192687 31901 ts_tablet_manager.cc:1918] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:40:22.192700 31935 ts_tablet_manager.cc:1918] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:40:22.192842 31935 log.cc:1196] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/7348c157b7e04351ba985325f06451a2
I20260926 20:40:22.193300 31935 ts_tablet_manager.cc:1939] T 7348c157b7e04351ba985325f06451a2 P c305b130115a46fd8a0dd27815c7dbbc: Deleting consensus metadata
I20260926 20:40:22.193516 31901 log.cc:1196] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/7348c157b7e04351ba985325f06451a2
I20260926 20:40:22.194015 31900 ts_tablet_manager.cc:1905] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:22.194056 31901 ts_tablet_manager.cc:1939] T 7348c157b7e04351ba985325f06451a2 P 69a1f31efdcc44e7a1dd36ea8e70c6d7: Deleting consensus metadata
I20260926 20:40:22.195333 31349 catalog_manager.cc:4841] TS 69a1f31efdcc44e7a1dd36ea8e70c6d7 (127.23.44.193:33983): tablet 7348c157b7e04351ba985325f06451a2 (table test-table [id=8a13b1bd76434a89ac888170e77d957d]) successfully deleted
I20260926 20:40:22.195708 31349 catalog_manager.cc:4841] TS c305b130115a46fd8a0dd27815c7dbbc (127.23.44.194:45199): tablet 7348c157b7e04351ba985325f06451a2 (table test-table [id=8a13b1bd76434a89ac888170e77d957d]) successfully deleted
I20260926 20:40:22.198938 31900 ts_tablet_manager.cc:1918] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 20:40:22.199133 31900 log.cc:1196] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/7348c157b7e04351ba985325f06451a2
I20260926 20:40:22.199571 31900 ts_tablet_manager.cc:1939] T 7348c157b7e04351ba985325f06451a2 P fea36b613dc84692831b4ea0079b8dd8: Deleting consensus metadata
I20260926 20:40:22.200623 31349 catalog_manager.cc:4841] TS fea36b613dc84692831b4ea0079b8dd8 (127.23.44.195:41185): tablet 7348c157b7e04351ba985325f06451a2 (table test-table [id=8a13b1bd76434a89ac888170e77d957d]) successfully deleted
Sep 26 20:40:22 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.44.193@KRBTEST.COM
I20260926 20:40:22.210538 31873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:22.188857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49054 (local address 127.23.44.193:33983)
0926 20:40:22.189059 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:22.189066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:22.193177 (+ 4111us) server_negotiation.cc:407] Connection header received
0926 20:40:22.193406 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:22.193411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:22.193517 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:22.193631 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:22.195651 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:22.196835 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:22.198195 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:22.198646 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:22.198787 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:22.203543 (+ 4756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:22.203566 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:22.203589 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:22.203645 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:22.207317 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:22.208112 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:22.208119 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:22.208122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:22.208264 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:22.209008 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:22.209013 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:22.209016 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:22.209391 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:22.209634 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:22.210033 (+ 399us) server_negotiation.cc:299] Negotiation successful
0926 20:40:22.210243 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 20:40:22.211603 31567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:22.339641 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 31436
I20260926 20:40:22.363124 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 31576
Sep 26 20:40:22 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.44.194@KRBTEST.COM
I20260926 20:40:22.380813 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:22.363877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36206 (local address 127.23.44.194:45199)
0926 20:40:22.364408 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:22.364413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:22.364428 (+ 15us) server_negotiation.cc:407] Connection header received
0926 20:40:22.364481 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:22.364486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:22.364547 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:22.364694 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:22.366126 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:22.366845 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:22.368165 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:22.368618 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:22.368804 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:22.373136 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:22.373152 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:22.373175 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:22.373481 (+ 306us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:22.377393 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:22.378257 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:22.378264 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:22.378267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:22.378413 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:22.378891 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:22.378897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:22.378900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:22.379412 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:22.379715 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:22.380164 (+ 449us) server_negotiation.cc:299] Negotiation successful
0926 20:40:22.380365 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":301,"threads_started":1}
I20260926 20:40:22.381846 31706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:22.508076 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 31576
I20260926 20:40:22.531721 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 31715
Sep 26 20:40:22 dist-test-slave-4d5t krb5kdc[31320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.44.195@KRBTEST.COM
I20260926 20:40:22.549784 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:22.532693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43714 (local address 127.23.44.195:41185)
0926 20:40:22.533150 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:22.533155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:22.533173 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:40:22.533342 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:22.533347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:22.533426 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:22.533549 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:22.535365 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:22.536385 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:22.537930 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:22.538365 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:22.538492 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:22.543166 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:22.543183 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:22.543200 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:22.543257 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:22.546358 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:22.547166 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:22.547174 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:22.547176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:22.547303 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:22.547797 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:22.547806 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:22.547808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:22.548400 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:22.548855 (+ 455us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:22.549289 (+ 434us) server_negotiation.cc:299] Negotiation successful
0926 20:40:22.549494 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":254,"threads_started":1}
I20260926 20:40:22.550484 31846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:22.656119 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 31715
I20260926 20:40:22.674902 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 31333
I20260926 20:40:22.676281 31405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:22.795553 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 31333
2026-09-26T20:40:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7000 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:40:22 dist-test-slave-4d5t krb5kdc[31973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:40:22 dist-test-slave-4d5t krb5kdc[31973](info): set up 2 sockets
Sep 26 20:40:22 dist-test-slave-4d5t krb5kdc[31973](info): commencing operation
krb5kdc: starting...
W20260926 20:40:23.950302 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:40:24 dist-test-slave-4d5t krb5kdc[31973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:40:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:40:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:40:24.199944 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:39589
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:39589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:24.535444 31986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:24.535811 31986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:24.535967 31986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:24.543520 31986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:40:24.543685 31986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:24.543802 31986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:24.543898 31986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:40:24.543995 31986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:40:24.554136 31986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:39589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:39589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:24.555538 31986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:24.556924 31986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:24.569757 31991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:24.569842 31992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:24.569837 31994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:24.571049 31986 server_base.cc:1034] running on GCE node
I20260926 20:40:24.571854 31986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:24.573397 31986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:24.574640 31986 hybrid_clock.cc:648] HybridClock initialized: now 1790455224574583 us; error 40 us; skew 500 ppm
I20260926 20:40:24.575070 31986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T20:40:24Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1853 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:40:24 dist-test-slave-4d5t krb5kdc[32000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:40:24 dist-test-slave-4d5t krb5kdc[32000](info): set up 2 sockets
Sep 26 20:40:24 dist-test-slave-4d5t krb5kdc[32000](info): commencing operation
krb5kdc: starting...
W20260926 20:40:25.790144 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:40:25 dist-test-slave-4d5t krb5kdc[32000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:40:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:40:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:40:26.034614 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33723
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:33723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:26.389178 32013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:26.389506 32013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:26.389678 32013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:26.397215 32013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:40:26.397349 32013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:26.397396 32013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:26.397486 32013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:40:26.397573 32013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:40:26.407691 32013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:33723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:26.409170 32013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:26.410503 32013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:26.423261 32018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:26.423231 32019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:26.423681 32021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:26.424247 32013 server_base.cc:1034] running on GCE node
I20260926 20:40:26.425318 32013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:26.427443 32013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:26.428761 32013 hybrid_clock.cc:648] HybridClock initialized: now 1790455226428687 us; error 50 us; skew 500 ppm
I20260926 20:40:26.429275 32013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:26 dist-test-slave-4d5t krb5kdc[32000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455226, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:26.433032 32013 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:40:26.434779 32013 webserver.cc:460] Webserver started at http://127.23.44.254:37027/ using document root <none> and password file <none>
I20260926 20:40:26.435546 32013 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:26.435700 32013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:26.436021 32013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:26.439118 32013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "9a2b2164cdbc47f6bf6c69efad090fea"
format_stamp: "Formatted at 2026-09-26 20:40:26 on dist-test-slave-4d5t"
server_key: "791592163130b5ddea2f959f6a866fd9"
server_key_iv: "4eddefe4327364ab9589339200410cc2"
server_key_version: "encryptionkey@0"
I20260926 20:40:26.439893 32013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "9a2b2164cdbc47f6bf6c69efad090fea"
format_stamp: "Formatted at 2026-09-26 20:40:26 on dist-test-slave-4d5t"
server_key: "791592163130b5ddea2f959f6a866fd9"
server_key_iv: "4eddefe4327364ab9589339200410cc2"
server_key_version: "encryptionkey@0"
I20260926 20:40:26.446298 32013 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260926 20:40:26.450435 32028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:26.451779 32013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 20:40:26.451992 32013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "9a2b2164cdbc47f6bf6c69efad090fea"
format_stamp: "Formatted at 2026-09-26 20:40:26 on dist-test-slave-4d5t"
server_key: "791592163130b5ddea2f959f6a866fd9"
server_key_iv: "4eddefe4327364ab9589339200410cc2"
server_key_version: "encryptionkey@0"
I20260926 20:40:26.452212 32013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:26.472667 32013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:26.478247 32013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:26.478592 32013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:26.499442 32013 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:33723
I20260926 20:40:26.499444 32079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:33723 every 8 connection(s)
I20260926 20:40:26.500969 32013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:40:26.505154 32080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:26.510357 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 32013
I20260926 20:40:26.510643 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:40:26.511534 23731 external_mini_cluster.cc:1428] Setting key 533fb83c1b1a9ff7c005bfb540ac45f3
I20260926 20:40:26.521234 32080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea: Bootstrap starting.
I20260926 20:40:26.524842 32080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:26.526268 32080 log.cc:824] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea: Log is configured to *not* fsync() on all Append() calls
Sep 26 20:40:26 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:40:26.530040 32080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea: No bootstrap required, opened a new log
I20260926 20:40:26.540051 32083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:26.514181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34946 (local address 127.23.44.254:33723)
0926 20:40:26.515226 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:26.515239 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:26.515283 (+ 44us) server_negotiation.cc:407] Connection header received
0926 20:40:26.517147 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:26.517173 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:26.517663 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:26.518133 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:26.519758 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:26.521427 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:26.522847 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:26.523683 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:26.523881 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:26.529600 (+ 5719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:26.529611 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:26.529637 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:26.529702 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:26.534549 (+ 4847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:26.535819 (+ 1270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:26.535829 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:26.535837 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:26.536087 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:26.536706 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:26.536711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:26.536713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:26.537622 (+ 909us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:26.537913 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:26.538406 (+ 493us) server_negotiation.cc:299] Negotiation successful
0926 20:40:26.538907 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":481,"threads_started":1}
I20260926 20:40:26.540907 32080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33723 } }
I20260926 20:40:26.541432 32080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:26.541599 32080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a2b2164cdbc47f6bf6c69efad090fea, State: Initialized, Role: FOLLOWER
I20260926 20:40:26.542215 32080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33723 } }
I20260926 20:40:26.542510 32080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:40:26.542680 32080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:40:26.542887 32080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:26.545488 32080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33723 } }
I20260926 20:40:26.546046 32080 leader_election.cc:302] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a2b2164cdbc47f6bf6c69efad090fea; no voters:
I20260926 20:40:26.546649 32080 leader_election.cc:288] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:40:26.546909 32085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:26.548125 32085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 1 LEADER]: Becoming Leader. State: Replica: 9a2b2164cdbc47f6bf6c69efad090fea, State: Running, Role: LEADER
I20260926 20:40:26.548722 32085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33723 } }
I20260926 20:40:26.549193 32080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:40:26.551921 32087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a2b2164cdbc47f6bf6c69efad090fea. Latest consensus state: current_term: 1 leader_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33723 } } }
I20260926 20:40:26.552402 32087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:26.552798 32086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33723 } } }
I20260926 20:40:26.553285 32086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:26.554262 32094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:40:26.561718 32094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:40:26.575057 32094 catalog_manager.cc:1347] Generated new cluster ID: 2bf48baf123e49b597d91038ddf02d62
I20260926 20:40:26.575186 32094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:40:26.586115 32094 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:40:26.587517 32094 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:40:26.595633 32094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea: Generated new TSK 0
I20260926 20:40:26.596937 32094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:40:26.668962 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33723
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:27.041198 32106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:27.041565 32106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:27.041759 32106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:27.050201 32106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:27.050449 32106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:27.050746 32106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:40:27.065277 32106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:27.066771 32106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:27.068257 32106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:27.082736 32111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:27.082698 32112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:27.082775 32114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:27.083813 32106 server_base.cc:1034] running on GCE node
I20260926 20:40:27.084656 32106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:27.085795 32106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:27.087081 32106 hybrid_clock.cc:648] HybridClock initialized: now 1790455227087041 us; error 66 us; skew 500 ppm
I20260926 20:40:27.087457 32106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:27 dist-test-slave-4d5t krb5kdc[32000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455227, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:27.091051 32106 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:40:27.092904 32106 webserver.cc:460] Webserver started at http://127.23.44.193:34609/ using document root <none> and password file <none>
I20260926 20:40:27.093693 32106 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:27.093863 32106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:27.094329 32106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:27.097606 32106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "7229fbf970fb43f397918d7d5083b666"
format_stamp: "Formatted at 2026-09-26 20:40:27 on dist-test-slave-4d5t"
server_key: "9eac2e422e7ec5cc90c331489dee62e1"
server_key_iv: "a1b21d6f0e5954d06fd05211ac045e82"
server_key_version: "encryptionkey@0"
I20260926 20:40:27.098412 32106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "7229fbf970fb43f397918d7d5083b666"
format_stamp: "Formatted at 2026-09-26 20:40:27 on dist-test-slave-4d5t"
server_key: "9eac2e422e7ec5cc90c331489dee62e1"
server_key_iv: "a1b21d6f0e5954d06fd05211ac045e82"
server_key_version: "encryptionkey@0"
I20260926 20:40:27.103839 32106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260926 20:40:27.107643 32121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:27.109344 32106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 20:40:27.109558 32106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "7229fbf970fb43f397918d7d5083b666"
format_stamp: "Formatted at 2026-09-26 20:40:27 on dist-test-slave-4d5t"
server_key: "9eac2e422e7ec5cc90c331489dee62e1"
server_key_iv: "a1b21d6f0e5954d06fd05211ac045e82"
server_key_version: "encryptionkey@0"
I20260926 20:40:27.109798 32106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:27.131325 32106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:27.136653 32106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:27.137032 32106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:27.138207 32106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:27.139727 32106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:27.139833 32106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:27.139993 32106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:27.140069 32106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:27.180124 32106 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:38829
I20260926 20:40:27.180253 32233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:38829 every 8 connection(s)
I20260926 20:40:27.181481 32106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:40:27.190766 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 32106
I20260926 20:40:27.191017 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:40:27.191660 23731 external_mini_cluster.cc:1428] Setting key b48604680454efe6bae91b62b7c448cb
Sep 26 20:40:27 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455227, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:40:27.205224 32236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:27.184681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:55409 (local address 127.23.44.254:33723)
0926 20:40:27.185125 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:27.185130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:27.186228 (+ 1098us) server_negotiation.cc:407] Connection header received
0926 20:40:27.188154 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:27.188159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:27.188263 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:27.188383 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:27.190029 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:27.190785 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:27.192684 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:27.193205 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:27.193320 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:27.197511 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:27.197521 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:27.197529 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:27.197577 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:27.201436 (+ 3859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:27.202589 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:27.202593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:27.202595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:27.202717 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:27.203282 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:27.203285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:27.203287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:27.203557 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:27.203664 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:27.204747 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0926 20:40:27.204922 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":223,"threads_started":1}
I20260926 20:40:27.206723 32237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:27.185246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:55409)
0926 20:40:27.186035 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:27.186075 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:27.187860 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:27.188658 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:27.188667 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:27.189229 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:27.189818 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:27.189836 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:27.190981 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:27.190992 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:27.192472 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:27.192500 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:27.193472 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:27.193479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:27.193832 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:27.194621 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:27.194650 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:27.197204 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:27.201681 (+ 4477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:27.201690 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:27.201706 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:27.202417 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:27.202833 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:27.202838 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:27.202844 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:27.203136 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:27.203693 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:27.203706 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:27.204216 (+ 510us) client_negotiation.cc:769] Sending connection context
0926 20:40:27.204543 (+ 327us) client_negotiation.cc:240] Negotiation successful
0926 20:40:27.205195 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":340,"threads_started":1}
I20260926 20:40:27.208441 32234 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33723
I20260926 20:40:27.208814 32234 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:27.209621 32234 heartbeater.cc:511] Master 127.23.44.254:33723 requested a full tablet report, sending...
I20260926 20:40:27.211820 32045 ts_manager.cc:194] Registered new tserver with Master: 7229fbf970fb43f397918d7d5083b666 (127.23.44.193:38829)
I20260926 20:40:27.213651 32045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:55409
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:40:27.248703 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33723
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:27.599391 32240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:27.599752 32240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:27.599920 32240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:27.607784 32240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:27.607978 32240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:27.608162 32240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:40:27.618516 32240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:27.619856 32240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:27.621222 32240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:27.635692 32245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:27.635823 32248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:27.635804 32246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:27.637024 32240 server_base.cc:1034] running on GCE node
I20260926 20:40:27.637715 32240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:27.638707 32240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:27.639932 32240 hybrid_clock.cc:648] HybridClock initialized: now 1790455227639861 us; error 61 us; skew 500 ppm
I20260926 20:40:27.640254 32240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:27 dist-test-slave-4d5t krb5kdc[32000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455227, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:27.643610 32240 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:40:27.645570 32240 webserver.cc:460] Webserver started at http://127.23.44.194:41227/ using document root <none> and password file <none>
I20260926 20:40:27.646267 32240 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:27.646376 32240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:27.646651 32240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:27.649839 32240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "c8329998fb644ff9bf3bf9f97b1fa391"
format_stamp: "Formatted at 2026-09-26 20:40:27 on dist-test-slave-4d5t"
server_key: "ad7121eadf77f0adb41f8f20e720524c"
server_key_iv: "83ea5f11ff823a3a1f3254f6034b7744"
server_key_version: "encryptionkey@0"
I20260926 20:40:27.650696 32240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "c8329998fb644ff9bf3bf9f97b1fa391"
format_stamp: "Formatted at 2026-09-26 20:40:27 on dist-test-slave-4d5t"
server_key: "ad7121eadf77f0adb41f8f20e720524c"
server_key_iv: "83ea5f11ff823a3a1f3254f6034b7744"
server_key_version: "encryptionkey@0"
I20260926 20:40:27.657847 32240 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260926 20:40:27.661932 32255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:27.663774 32240 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:40:27.663939 32240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "c8329998fb644ff9bf3bf9f97b1fa391"
format_stamp: "Formatted at 2026-09-26 20:40:27 on dist-test-slave-4d5t"
server_key: "ad7121eadf77f0adb41f8f20e720524c"
server_key_iv: "83ea5f11ff823a3a1f3254f6034b7744"
server_key_version: "encryptionkey@0"
I20260926 20:40:27.664139 32240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:27.680847 32240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:27.686645 32240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:27.686916 32240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:27.687945 32240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:27.689375 32240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:27.689450 32240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:27.689548 32240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:27.689592 32240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:27.729987 32240 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:45617
I20260926 20:40:27.730006 32367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:45617 every 8 connection(s)
I20260926 20:40:27.731346 32240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:40:27.734929 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 32240
I20260926 20:40:27.735121 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:40:27.735703 23731 external_mini_cluster.cc:1428] Setting key 875b0bc0f55dda879e35a50acd0a7866
Sep 26 20:40:27 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455227, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:40:27.755255 32370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:27.734177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:58217 (local address 127.23.44.254:33723)
0926 20:40:27.734743 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:27.734751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:27.735994 (+ 1243us) server_negotiation.cc:407] Connection header received
0926 20:40:27.738238 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:27.738243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:27.738313 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:27.738407 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:27.740671 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:27.741428 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:27.743977 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:27.744430 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:27.744545 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:27.748472 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:27.748477 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:27.748480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:27.748514 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:27.751507 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:27.752430 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:27.752436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:27.752439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:27.752664 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:27.753133 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:27.753141 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:27.753147 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:27.753560 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:27.753739 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:27.754508 (+ 769us) server_negotiation.cc:299] Negotiation successful
0926 20:40:27.754770 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":284,"threads_started":1}
I20260926 20:40:27.756019 32371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:27.734838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:58217)
0926 20:40:27.735794 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:27.735850 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:27.737915 (+ 2065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:27.738713 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:27.738724 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:27.739554 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:27.740399 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:27.740425 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:27.741602 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:27.741609 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:27.743800 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:27.743818 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:27.744704 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:27.744714 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:27.745050 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:27.745820 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:27.745858 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:27.748227 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:27.751682 (+ 3455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:27.751692 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:27.751713 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:27.752301 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:27.752777 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:27.752782 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:27.752786 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:27.752981 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:27.753752 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:27.753758 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:27.754136 (+ 378us) client_negotiation.cc:769] Sending connection context
0926 20:40:27.754384 (+ 248us) client_negotiation.cc:240] Negotiation successful
0926 20:40:27.754788 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":395,"threads_started":1}
I20260926 20:40:27.757694 32368 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33723
I20260926 20:40:27.758052 32368 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:27.758844 32368 heartbeater.cc:511] Master 127.23.44.254:33723 requested a full tablet report, sending...
I20260926 20:40:27.760438 32045 ts_manager.cc:194] Registered new tserver with Master: c8329998fb644ff9bf3bf9f97b1fa391 (127.23.44.194:45617)
I20260926 20:40:27.761296 32045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:58217
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:40:27.792532 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33723
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:28.150955 32374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:28.151307 32374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:28.151481 32374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:28.159246 32374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:28.159423 32374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:28.159597 32374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:40:28.170087 32374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:28.171507 32374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:28.172922 32374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:28.190237 32379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:28.190207 32380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:28.190165 32382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:28.191177 32374 server_base.cc:1034] running on GCE node
I20260926 20:40:28.191907 32374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:28.193054 32374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:28.194320 32374 hybrid_clock.cc:648] HybridClock initialized: now 1790455228194272 us; error 59 us; skew 500 ppm
I20260926 20:40:28.194687 32374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:28 dist-test-slave-4d5t krb5kdc[32000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455228, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:28.198136 32374 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:40:28.199785 32374 webserver.cc:460] Webserver started at http://127.23.44.195:46477/ using document root <none> and password file <none>
I20260926 20:40:28.200480 32374 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:28.200647 32374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:28.200927 32374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:28.203809 32374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "6fe426899fe048f4aa6a7606283a9808"
format_stamp: "Formatted at 2026-09-26 20:40:28 on dist-test-slave-4d5t"
server_key: "8ef1a6960a0108ae2528c1cf5d75eb09"
server_key_iv: "2b2daa5ebf4919e004b5f011446ee74c"
server_key_version: "encryptionkey@0"
I20260926 20:40:28.204622 32374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "6fe426899fe048f4aa6a7606283a9808"
format_stamp: "Formatted at 2026-09-26 20:40:28 on dist-test-slave-4d5t"
server_key: "8ef1a6960a0108ae2528c1cf5d75eb09"
server_key_iv: "2b2daa5ebf4919e004b5f011446ee74c"
server_key_version: "encryptionkey@0"
I20260926 20:40:28.210282 32374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260926 20:40:28.214092 32389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:28.215564 32374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260926 20:40:28.215740 32374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "6fe426899fe048f4aa6a7606283a9808"
format_stamp: "Formatted at 2026-09-26 20:40:28 on dist-test-slave-4d5t"
server_key: "8ef1a6960a0108ae2528c1cf5d75eb09"
server_key_iv: "2b2daa5ebf4919e004b5f011446ee74c"
server_key_version: "encryptionkey@0"
I20260926 20:40:28.215965 32374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:28.217235 32234 heartbeater.cc:503] Master 127.23.44.254:33723 was elected leader, sending a full tablet report...
I20260926 20:40:28.243276 32374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:28.247967 32374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:28.248281 32374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:28.249346 32374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:28.250803 32374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:28.250908 32374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:28.251057 32374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:28.251137 32374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:28.287106 32374 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:42637
I20260926 20:40:28.287132 32501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:42637 every 8 connection(s)
I20260926 20:40:28.288344 32374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:40:28.298156 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 32374
I20260926 20:40:28.298365 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:40:28.298810 23731 external_mini_cluster.cc:1428] Setting key a4db8cbc202b22840f02ebe5775fc123
Sep 26 20:40:28 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455228, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:40:28.309113 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:28.290876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:39097 (local address 127.23.44.254:33723)
0926 20:40:28.291309 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:28.291314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:28.292223 (+ 909us) server_negotiation.cc:407] Connection header received
0926 20:40:28.294033 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:28.294037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:28.294156 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:28.294268 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:28.295678 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:28.296476 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:28.298321 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:28.298816 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:28.298968 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:28.302774 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:28.302787 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:28.302791 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:28.302835 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:28.305645 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:28.306425 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:28.306429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:28.306431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:28.306552 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:28.307373 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:28.307376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:28.307378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:28.307657 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:28.307777 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:28.308676 (+ 899us) server_negotiation.cc:299] Negotiation successful
0926 20:40:28.308846 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":217,"threads_started":1}
I20260926 20:40:28.310127 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:28.291372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:39097)
0926 20:40:28.292055 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:28.292094 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:28.293795 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:28.294466 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:28.294474 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:28.294968 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:28.295508 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:28.295525 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:28.296684 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:28.296688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:28.298100 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:28.298122 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:28.299079 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:28.299089 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:28.299378 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:28.300133 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:28.300155 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:28.302357 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:28.305815 (+ 3458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:28.305822 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:28.305838 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:28.306288 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:28.306754 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:28.306760 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:28.306764 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:28.307185 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:28.307820 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:28.307827 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:28.308205 (+ 378us) client_negotiation.cc:769] Sending connection context
0926 20:40:28.308459 (+ 254us) client_negotiation.cc:240] Negotiation successful
0926 20:40:28.308970 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":273,"threads_started":1}
I20260926 20:40:28.312350 32502 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33723
I20260926 20:40:28.312980 32502 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:28.314087 32502 heartbeater.cc:511] Master 127.23.44.254:33723 requested a full tablet report, sending...
I20260926 20:40:28.316167 32045 ts_manager.cc:194] Registered new tserver with Master: 6fe426899fe048f4aa6a7606283a9808 (127.23.44.195:42637)
I20260926 20:40:28.317050 32045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:39097
I20260926 20:40:28.326520 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab.
Entry for principal oryx/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab.
WARNING: no policy specified for oryx/127.23.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab.
Entry for principal oryx/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab.
WARNING: no policy specified for oryx/127.23.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab.
Entry for principal oryx/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab.
I20260926 20:40:28.484434 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 32106
Sep 26 20:40:28 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:40:28.503492 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:28.485672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39200 (local address 127.23.44.193:38829)
0926 20:40:28.486347 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:28.486354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:28.486382 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:40:28.486444 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:28.486452 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:28.486700 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:28.486850 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:28.488351 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:28.490103 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:28.491563 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:28.492234 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:28.492361 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:28.497147 (+ 4786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:28.497162 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:28.497186 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:28.497229 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:28.500427 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:28.501376 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:28.501385 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:28.501388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:28.501534 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:28.501937 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:28.501941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:28.501943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:28.502271 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:28.502503 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:28.502981 (+ 478us) server_negotiation.cc:299] Negotiation successful
0926 20:40:28.503175 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":368,"threads_started":1}
I20260926 20:40:28.504868 32229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:28.588546 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 32106
I20260926 20:40:28.605733 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 32240
Sep 26 20:40:28 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:40:28.623498 32515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:28.607177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43702 (local address 127.23.44.194:45617)
0926 20:40:28.607973 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:28.607979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:28.608006 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:40:28.608071 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:28.608078 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:28.608288 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:28.608426 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:28.609810 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:28.610836 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:28.612073 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:28.612844 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:28.612962 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:28.617453 (+ 4491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:28.617466 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:28.617481 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:28.617517 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:28.620638 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:28.621454 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:28.621466 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:28.621471 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:28.621656 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:28.622134 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:28.622138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:28.622140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:28.622476 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:28.622710 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:28.623010 (+ 300us) server_negotiation.cc:299] Negotiation successful
0926 20:40:28.623209 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":548,"threads_started":1}
I20260926 20:40:28.624889 32363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:28.712702 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 32240
I20260926 20:40:28.728632 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 32374
Sep 26 20:40:28 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:40:28.746381 32517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:28.729742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53468 (local address 127.23.44.195:42637)
0926 20:40:28.730341 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:28.730348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:28.730380 (+ 32us) server_negotiation.cc:407] Connection header received
0926 20:40:28.730444 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:28.730452 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:28.730675 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:28.730810 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:28.732537 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:28.733661 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:28.734938 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:28.735578 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:28.735696 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:28.740347 (+ 4651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:28.740361 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:28.740378 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:28.740417 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:28.743362 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:28.744131 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:28.744139 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:28.744141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:28.744261 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:28.744799 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:28.744802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:28.744804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:28.745163 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:28.745393 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:28.745707 (+ 314us) server_negotiation.cc:299] Negotiation successful
0926 20:40:28.745913 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":296,"threads_started":1}
I20260926 20:40:28.747778 32497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:28.836441 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 32374
I20260926 20:40:28.853008 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 32013
I20260926 20:40:28.854460 32075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:28.959806 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 32013
I20260926 20:40:28.981845 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33723
--webserver_interface=127.23.44.254
--webserver_port=37027
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:33723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:29.338937 32520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:29.339286 32520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:29.339483 32520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:29.347498 32520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:40:29.347625 32520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:29.347721 32520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:29.347810 32520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:40:29.347894 32520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:40:29.357970 32520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:33723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=37027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:29.359330 32520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:29.360649 32520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:29.373800 32525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:29.373718 32528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:29.373876 32526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:29.374837 32520 server_base.cc:1034] running on GCE node
I20260926 20:40:29.375686 32520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:29.377321 32520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:29.378597 32520 hybrid_clock.cc:648] HybridClock initialized: now 1790455229378548 us; error 70 us; skew 500 ppm
I20260926 20:40:29.379029 32520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:29 dist-test-slave-4d5t krb5kdc[32000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455229, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:29.382431 32520 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:40:29.384063 32520 webserver.cc:460] Webserver started at http://127.23.44.254:37027/ using document root <none> and password file <none>
I20260926 20:40:29.384888 32520 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:29.385051 32520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:29.391294 32520 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:40:29.394977 32535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:29.396543 32520 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 20:40:29.396848 32520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "9a2b2164cdbc47f6bf6c69efad090fea"
format_stamp: "Formatted at 2026-09-26 20:40:26 on dist-test-slave-4d5t"
server_key: "791592163130b5ddea2f959f6a866fd9"
server_key_iv: "4eddefe4327364ab9589339200410cc2"
server_key_version: "encryptionkey@0"
I20260926 20:40:29.397945 32520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:29.415297 32520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:29.420413 32520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:29.420761 32520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:29.442860 32520 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:33723
I20260926 20:40:29.442889 32586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:33723 every 8 connection(s)
I20260926 20:40:29.444380 32520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:40:29.449676 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 32520
I20260926 20:40:29.450475 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:38829
--local_ip_for_outbound_sockets=127.23.44.193
--tserver_master_addrs=127.23.44.254:33723
--webserver_port=34609
--webserver_interface=127.23.44.193
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.44.193
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:40:29.451786 32587 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:40:29.455264 32587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea: Bootstrap starting.
I20260926 20:40:29.476456 32587 log.cc:824] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:29.485325 32587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:40:29.485869 32587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea: Bootstrap complete.
I20260926 20:40:29.494941 32587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33723 } }
I20260926 20:40:29.495857 32587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a2b2164cdbc47f6bf6c69efad090fea, State: Initialized, Role: FOLLOWER
I20260926 20:40:29.496528 32587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33723 } }
I20260926 20:40:29.496867 32587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:40:29.496997 32587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:40:29.497189 32587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 1 FOLLOWER]: Advancing to term 2
I20260926 20:40:29.501156 32587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33723 } }
I20260926 20:40:29.501797 32587 leader_election.cc:302] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a2b2164cdbc47f6bf6c69efad090fea; no voters:
I20260926 20:40:29.502467 32587 leader_election.cc:288] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 20:40:29.502621 32591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 2 FOLLOWER]: Leader election won for term 2
I20260926 20:40:29.504026 32591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [term 2 LEADER]: Becoming Leader. State: Replica: 9a2b2164cdbc47f6bf6c69efad090fea, State: Running, Role: LEADER
I20260926 20:40:29.504689 32591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33723 } }
I20260926 20:40:29.505142 32587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:40:29.507398 32593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a2b2164cdbc47f6bf6c69efad090fea. Latest consensus state: current_term: 2 leader_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33723 } } }
I20260926 20:40:29.507772 32593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:29.508344 32592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a2b2164cdbc47f6bf6c69efad090fea" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33723 } } }
I20260926 20:40:29.508910 32592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:29.517634 32596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:40:29.525805 32596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:40:29.527948 32596 catalog_manager.cc:1259] Loaded cluster ID: 2bf48baf123e49b597d91038ddf02d62
I20260926 20:40:29.528107 32596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:40:29.534880 32596 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:40:29.537115 32596 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9a2b2164cdbc47f6bf6c69efad090fea: Loaded TSK: 0
I20260926 20:40:29.539069 32596 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 20:40:29.830788 32589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:29.831120 32589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:29.831266 32589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:29.839767 32589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:29.839900 32589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:29.840205 32589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:40:29.853313 32589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.193.keytab
--principal=oryx/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:38829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=34609
--tserver_master_addrs=127.23.44.254:33723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:29.855093 32589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:29.856613 32589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:29.872898 32614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:29.873052 32613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:29.873515 32616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:29.873907 32589 server_base.cc:1034] running on GCE node
I20260926 20:40:29.874682 32589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:29.875924 32589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:29.877231 32589 hybrid_clock.cc:648] HybridClock initialized: now 1790455229877122 us; error 68 us; skew 500 ppm
I20260926 20:40:29.877772 32589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:29 dist-test-slave-4d5t krb5kdc[32000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455229, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:29.886335 32589 init.cc:377] Logged in from keytab as oryx/127.23.44.193@KRBTEST.COM (short username oryx)
I20260926 20:40:29.888190 32589 webserver.cc:460] Webserver started at http://127.23.44.193:34609/ using document root <none> and password file <none>
I20260926 20:40:29.889433 32589 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:29.889580 32589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:29.895919 32589 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 20:40:29.899034 32623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:29.900878 32589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:40:29.901074 32589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "7229fbf970fb43f397918d7d5083b666"
format_stamp: "Formatted at 2026-09-26 20:40:27 on dist-test-slave-4d5t"
server_key: "9eac2e422e7ec5cc90c331489dee62e1"
server_key_iv: "a1b21d6f0e5954d06fd05211ac045e82"
server_key_version: "encryptionkey@0"
I20260926 20:40:29.902215 32589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:29.932400 32589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:29.938441 32589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:29.938766 32589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:29.939826 32589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:29.941375 32589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:29.941470 32589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:29.941601 32589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:29.941702 32589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:29.980991 32589 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:38829
I20260926 20:40:29.981050 32735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:38829 every 8 connection(s)
I20260926 20:40:29.982607 32589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:40:29.986640 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 32589
I20260926 20:40:29.987416 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:45617
--local_ip_for_outbound_sockets=127.23.44.194
--tserver_master_addrs=127.23.44.254:33723
--webserver_port=41227
--webserver_interface=127.23.44.194
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.44.194
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:40:29 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:29 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 20:40:30.345438 32740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:30.345825 32740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:30.346007 32740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:30.354979 32740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:30.355175 32740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:30.355366 32740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:40:30.366693 32740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.194.keytab
--principal=oryx/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:45617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=41227
--tserver_master_addrs=127.23.44.254:33723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:30.368211 32740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:30.369721 32740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:30.389871 32746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:30.389871 32747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:30.390734 32749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:30.391894 32740 server_base.cc:1034] running on GCE node
I20260926 20:40:30.392652 32740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:30.393821 32740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:30.395298 32740 hybrid_clock.cc:648] HybridClock initialized: now 1790455230395201 us; error 94 us; skew 500 ppm
I20260926 20:40:30.395763 32740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:30 dist-test-slave-4d5t krb5kdc[32000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455230, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:30.399959 32740 init.cc:377] Logged in from keytab as oryx/127.23.44.194@KRBTEST.COM (short username oryx)
I20260926 20:40:30.401803 32740 webserver.cc:460] Webserver started at http://127.23.44.194:41227/ using document root <none> and password file <none>
I20260926 20:40:30.402598 32740 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:30.402782 32740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:30.409461 32740 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.003s
I20260926 20:40:30.412635 32756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:30.414489 32740 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260926 20:40:30.414683 32740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "c8329998fb644ff9bf3bf9f97b1fa391"
format_stamp: "Formatted at 2026-09-26 20:40:27 on dist-test-slave-4d5t"
server_key: "ad7121eadf77f0adb41f8f20e720524c"
server_key_iv: "83ea5f11ff823a3a1f3254f6034b7744"
server_key_version: "encryptionkey@0"
I20260926 20:40:30.415664 32740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:30.457592 32740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:30.462332 32740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:30.462682 32740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:30.463876 32740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:30.465723 32740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:30.465865 32740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:30.466014 32740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:30.466128 32740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:30.524058 32740 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:45617
I20260926 20:40:30.524243 400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:45617 every 8 connection(s)
I20260926 20:40:30.525763 32740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:40:30.526823 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 32740
I20260926 20:40:30.527908 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:42637
--local_ip_for_outbound_sockets=127.23.44.195
--tserver_master_addrs=127.23.44.254:33723
--webserver_port=46477
--webserver_interface=127.23.44.195
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.44.195
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:40:30 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:30 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 20:40:31.149134 32739 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.150s user 0.002s sys 0.018s
W20260926 20:40:31.151621 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:29.985703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:36881 (local address 127.23.44.254:33723)
0926 20:40:29.986545 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:29.986562 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:29.987094 (+ 532us) server_negotiation.cc:407] Connection header received
0926 20:40:29.990388 (+ 3294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:29.990400 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:29.990751 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:29.991084 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:29.993298 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:29.995058 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:29.996752 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:29.997411 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:29.997648 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:31.150904 (+1153256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:36881: BlockingRecv error: recv got EOF from 127.23.44.193:36881 (error 108)
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":286,"threads_started":1}
I20260926 20:40:31.151631 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:29.985790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:36881)
0926 20:40:29.986887 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:29.986942 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:29.989755 (+ 2813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:29.991309 (+ 1554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:29.991322 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:29.992149 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:29.993025 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:29.993055 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:29.995265 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:29.995270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:29.996531 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:29.996545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:29.997686 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:29.997694 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:29.997989 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:29.998828 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:29.998867 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:31.150481 (+1151614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":587,"threads_started":1}
W20260926 20:40:31.152874 32736 heartbeater.cc:644] Failed to heartbeat to 127.23.44.254:33723 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.44.254:33723: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Sep 26 20:40:31 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:31 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 20:40:31.165082 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:31.153703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:42481 (local address 127.23.44.254:33723)
0926 20:40:31.153883 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:31.153891 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:31.153958 (+ 67us) server_negotiation.cc:407] Connection header received
0926 20:40:31.154338 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:31.154349 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:31.154481 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:31.154654 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:31.155589 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.156902 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.158653 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.159103 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.159266 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:31.164844 (+ 5578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:42481: BlockingRecv error: recv got EOF from 127.23.44.193:42481 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 20:40:31.165050 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:31.153627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:42481)
0926 20:40:31.153883 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:31.153901 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:31.154035 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:31.154695 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:31.154700 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:31.155215 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:31.155411 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:31.155422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.157186 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.157192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:31.158384 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:31.158402 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.159332 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.159339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:31.159539 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:31.160293 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:31.160315 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:31.164444 (+ 4129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149}
Sep 26 20:40:31 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:31 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:31.175800 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:31.166410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:36451)
0926 20:40:31.166738 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:31.166753 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:31.166922 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:31.167308 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:31.167312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:31.167709 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:31.167893 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:31.167903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.169153 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.169158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:31.170157 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:31.170168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.170885 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.170891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:31.171088 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:31.171871 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:31.171890 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:31.175510 (+ 3620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241}
W20260926 20:40:31.175976 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:31.166457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:36451 (local address 127.23.44.254:33723)
0926 20:40:31.166631 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:31.166636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:31.166843 (+ 207us) server_negotiation.cc:407] Connection header received
0926 20:40:31.167064 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:31.167070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:31.167155 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:31.167282 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:31.168042 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.168997 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.170307 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.170728 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.170870 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:31.175740 (+ 4870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:36451: BlockingRecv error: recv got EOF from 127.23.44.193:36451 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 20:40:31.176466 32736 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 20:40:31.249560 405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:31.249840 405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:31.249992 405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:31.257431 405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:31.257560 405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:31.257730 405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:40:31.267688 405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/krb5kdc/oryx_127.23.44.195.keytab
--principal=oryx/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:42637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=46477
--tserver_master_addrs=127.23.44.254:33723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:31.269191 405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:31.270555 405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:31.291505 412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:31.292156 411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:31.291671 414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:31.293283 405 server_base.cc:1034] running on GCE node
I20260926 20:40:31.294035 405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:31.295264 405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:31.296705 405 hybrid_clock.cc:648] HybridClock initialized: now 1790455231296619 us; error 85 us; skew 500 ppm
I20260926 20:40:31.297060 405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:31 dist-test-slave-4d5t krb5kdc[32000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455231, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:31.301839 405 init.cc:377] Logged in from keytab as oryx/127.23.44.195@KRBTEST.COM (short username oryx)
I20260926 20:40:31.303903 405 webserver.cc:460] Webserver started at http://127.23.44.195:46477/ using document root <none> and password file <none>
I20260926 20:40:31.304704 405 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:31.304857 405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:31.312764 405 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.004s
I20260926 20:40:31.317276 421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:31.319454 405 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:40:31.319676 405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "6fe426899fe048f4aa6a7606283a9808"
format_stamp: "Formatted at 2026-09-26 20:40:28 on dist-test-slave-4d5t"
server_key: "8ef1a6960a0108ae2528c1cf5d75eb09"
server_key_iv: "2b2daa5ebf4919e004b5f011446ee74c"
server_key_version: "encryptionkey@0"
I20260926 20:40:31.320987 405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:31.335798 405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:31.340516 405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:31.340896 405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:31.342133 405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:31.343616 405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:31.343748 405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:31.343878 405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:31.343972 405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:31.385752 405 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:42637
I20260926 20:40:31.385803 533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:42637 every 8 connection(s)
I20260926 20:40:31.387387 405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:40:31.394686 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 405
Sep 26 20:40:31 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:31 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:31.419634 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:31.395912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34956 (local address 127.23.44.254:33723)
0926 20:40:31.396450 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:31.396456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:31.397170 (+ 714us) server_negotiation.cc:407] Connection header received
0926 20:40:31.397332 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:31.397341 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:31.397473 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:31.397630 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:31.399285 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.400176 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.402143 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.403250 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.403596 (+ 346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:31.409387 (+ 5791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:31.409401 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:31.409429 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:31.409504 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:31.414282 (+ 4778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:31.415574 (+ 1292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:31.415586 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:31.415599 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:31.415831 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:31.416391 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:31.416399 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:31.416406 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:31.417419 (+ 1013us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:31.417789 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:31.418519 (+ 730us) server_negotiation.cc:299] Negotiation successful
0926 20:40:31.419047 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":271,"server-negotiator.queue_time_us":424,"thread_start_us":307,"threads_started":1}
W20260926 20:40:31.837700 404 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.279s user 0.007s sys 0.014s
W20260926 20:40:31.839630 403 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:30.531439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:56001 (local address 127.23.44.254:33723)
0926 20:40:30.541019 (+ 9580us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:30.541026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:30.541609 (+ 583us) server_negotiation.cc:407] Connection header received
0926 20:40:30.546179 (+ 4570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:30.546189 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:30.546308 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:30.546452 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:30.549919 (+ 3467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:30.551339 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:30.555537 (+ 4198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:30.556428 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:30.556667 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:31.839349 (+1282682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:56001: BlockingRecv error: recv got EOF from 127.23.44.194:56001 (error 108)
Metrics: {"server-negotiator.queue_time_us":9406,"thread_start_us":9251,"threads_started":1}
I20260926 20:40:31.841574 404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:30.532367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:56001)
0926 20:40:30.541325 (+ 8958us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:30.541423 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:30.545651 (+ 4228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:30.546913 (+ 1262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:30.546933 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:30.548198 (+ 1265us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:30.549571 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:30.549609 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:30.551606 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:30.551615 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:30.555242 (+ 3627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:30.555273 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:30.556807 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:30.556818 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:30.557303 (+ 485us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:30.558486 (+ 1183us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:30.558546 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:31.839182 (+1280636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":8472,"thread_start_us":8350,"threads_started":1}
W20260926 20:40:31.842648 401 heartbeater.cc:644] Failed to heartbeat to 127.23.44.254:33723 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.44.254:33723: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Sep 26 20:40:31 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:31 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:31.857122 404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:31.843281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:58719)
0926 20:40:31.843590 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:31.843622 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:31.843823 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:31.844217 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:31.844226 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:31.845056 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:31.845395 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:31.845415 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.846755 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.846765 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:31.849381 (+ 2616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:31.849406 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.850636 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.850647 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:31.850981 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:31.851920 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:31.851960 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:31.856897 (+ 4937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93}
W20260926 20:40:31.857745 403 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:31.843414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:58719 (local address 127.23.44.254:33723)
0926 20:40:31.843580 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:31.843584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:31.843716 (+ 132us) server_negotiation.cc:407] Connection header received
0926 20:40:31.843981 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:31.843985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:31.844044 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:31.844120 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:31.845673 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.846573 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.849753 (+ 3180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.850316 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.850480 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:31.857353 (+ 6873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:58719: BlockingRecv error: recv got EOF from 127.23.44.194:58719 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 20:40:31 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:31 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:31.867789 404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:31.857891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:45661)
0926 20:40:31.858244 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:31.858264 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:31.858380 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:31.858724 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:31.858728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:31.859072 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:31.859230 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:31.859238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.860273 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.860278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:31.861337 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:31.861349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.862104 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.862110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:31.862301 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:31.863149 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:31.863171 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:31.867529 (+ 4358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249}
W20260926 20:40:31.868131 401 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 20:40:31.868774 403 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:31.858081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:45661 (local address 127.23.44.254:33723)
0926 20:40:31.858240 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:31.858247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:31.858316 (+ 69us) server_negotiation.cc:407] Connection header received
0926 20:40:31.858520 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:31.858525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:31.858598 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:31.858700 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:31.859370 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.860126 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.861485 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.861951 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.862090 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:31.868355 (+ 6265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:45661: BlockingRecv error: recv got EOF from 127.23.44.194:45661 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 26 20:40:32 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:32 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:32.190062 540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:32.177179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:41147)
0926 20:40:32.177972 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:32.178013 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:32.178157 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:32.178650 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:32.178656 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:32.179185 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:32.179329 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:32.179338 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.181387 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.181391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:32.182430 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:32.182442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.183829 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.183834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:32.183966 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:32.184994 (+ 1028us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:32.185022 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:32.189461 (+ 4439us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":538,"threads_started":1}
W20260926 20:40:32.190353 403 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:32.177245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:41147 (local address 127.23.44.254:33723)
0926 20:40:32.177507 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:32.177517 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:32.178122 (+ 605us) server_negotiation.cc:407] Connection header received
0926 20:40:32.178302 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:32.178312 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:32.178467 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:32.178630 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:32.179492 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.181150 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.182593 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.183570 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.183769 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:32.189988 (+ 6219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:41147: BlockingRecv error: recv got EOF from 127.23.44.193:41147 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260926 20:40:32.552757 536 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.148s user 0.010s sys 0.007s
W20260926 20:40:32.554129 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:31.390771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:38305 (local address 127.23.44.254:33723)
0926 20:40:31.391017 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:31.391022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:31.392602 (+ 1580us) server_negotiation.cc:407] Connection header received
0926 20:40:31.395677 (+ 3075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:31.395683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:31.395773 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:31.395861 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:31.398567 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.400246 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.402530 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.403263 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.403417 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:32.553886 (+1150469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:38305: BlockingRecv error: recv got EOF from 127.23.44.195:38305 (error 108)
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":93}
I20260926 20:40:32.554692 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:31.391449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:38305)
0926 20:40:31.392392 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:31.392439 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:31.395308 (+ 2869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:31.396671 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:31.396688 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:31.397487 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:31.398358 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:31.398385 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.400456 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.400461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:31.402387 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:31.402404 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:31.403532 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:31.403540 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:31.403826 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:31.404571 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:31.404606 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:32.553714 (+1149108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":424,"threads_started":1}
W20260926 20:40:32.555655 534 heartbeater.cc:644] Failed to heartbeat to 127.23.44.254:33723 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.44.254:33723: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Sep 26 20:40:32 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:32 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:32.565649 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:32.556397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:54647)
0926 20:40:32.556643 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:32.556660 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:32.556760 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:32.557069 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:32.557072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:32.557433 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:32.557617 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:32.557625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.558771 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.558775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:32.559728 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:32.559739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.560645 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.560651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:32.560773 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:32.561452 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:32.561470 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:32.565416 (+ 3946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260926 20:40:32.565879 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:32.556463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:54647 (local address 127.23.44.254:33723)
0926 20:40:32.556722 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:32.556726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:32.556745 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:40:32.556887 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:32.556890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:32.556950 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:32.557030 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:32.557772 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.558629 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.559866 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.560398 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.560503 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:32.565641 (+ 5138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:54647: BlockingRecv error: recv got EOF from 127.23.44.195:54647 (error 108)
Metrics: {"server-negotiator.queue_time_us":164}
Sep 26 20:40:32 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:32 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:32.576428 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:32.566369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:42025)
0926 20:40:32.566547 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:32.566565 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:32.566678 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:32.567040 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:32.567044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:32.567391 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:32.567556 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:32.567565 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.568587 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.568593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:32.569936 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:32.569949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.571013 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.571018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:32.571222 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:32.571882 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:32.571908 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:32.576129 (+ 4221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 20:40:32.576709 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:32.566440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:42025 (local address 127.23.44.254:33723)
0926 20:40:32.566617 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:32.566623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:32.566641 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:40:32.566813 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:32.566818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:32.566904 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:32.567020 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:32.567695 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.568415 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.570099 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.570828 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.571013 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:32.576332 (+ 5319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:42025: BlockingRecv error: recv got EOF from 127.23.44.195:42025 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 20:40:32.576884 534 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 20:40:32 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:32 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:32.878252 541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:32.868967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:59815)
0926 20:40:32.869588 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:32.869609 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:32.869760 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:32.870192 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:32.870197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:32.870712 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:32.870859 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:32.870867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.871898 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.871902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:32.872888 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:32.872901 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.873671 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.873678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:32.873884 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:32.874590 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:32.874608 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:32.878013 (+ 3405us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":339,"threads_started":1}
W20260926 20:40:32.878608 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:32.868998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:59815 (local address 127.23.44.254:33723)
0926 20:40:32.869226 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:32.869236 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:32.869749 (+ 513us) server_negotiation.cc:407] Connection header received
0926 20:40:32.869889 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:32.869895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:32.870016 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:32.870103 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:32.871030 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.871750 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.873027 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:32.873439 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:32.873545 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:32.878271 (+ 4726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:59815: BlockingRecv error: recv got EOF from 127.23.44.194:59815 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 26 20:40:33 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:33 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:33.201740 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:33.191273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:35873)
0926 20:40:33.191818 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:33.191840 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:33.192013 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:33.192363 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:33.192368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:33.192856 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:33.193052 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:33.193062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:33.194086 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:33.194091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:33.195021 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:33.195032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:33.195789 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:33.195795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:33.195977 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:33.196678 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:33.196701 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:33.201285 (+ 4584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":319,"threads_started":1}
W20260926 20:40:33.201817 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:33.191324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:35873 (local address 127.23.44.254:33723)
0926 20:40:33.191552 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:33.191560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:33.191946 (+ 386us) server_negotiation.cc:407] Connection header received
0926 20:40:33.192144 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:33.192149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:33.192220 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:33.192333 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:33.193202 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:33.193947 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:33.195172 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:33.195620 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:33.195753 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:33.201508 (+ 5755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:35873: BlockingRecv error: recv got EOF from 127.23.44.193:35873 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 20:40:33 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:33 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:33.589318 543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:33.577878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:35977)
0926 20:40:33.578571 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:33.578595 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:33.578794 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:33.579230 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:33.579237 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:33.579728 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:33.579889 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:33.579898 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:33.581462 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:33.581468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:33.582827 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:33.582841 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:33.583762 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:33.583768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:33.583913 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:33.584661 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:33.584681 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:33.588942 (+ 4261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":403,"threads_started":1}
W20260926 20:40:33.589429 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:33.578001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:35977 (local address 127.23.44.254:33723)
0926 20:40:33.578229 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:33.578239 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:33.578711 (+ 472us) server_negotiation.cc:407] Connection header received
0926 20:40:33.578949 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:33.578955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:33.579062 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:33.579192 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:33.580057 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:33.581301 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:33.582988 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:33.583509 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:33.583682 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:33.589180 (+ 5498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:35977: BlockingRecv error: recv got EOF from 127.23.44.195:35977 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 26 20:40:33 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:33 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:33.889254 544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:33.879250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:47373)
0926 20:40:33.879760 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:33.879776 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:33.879943 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:33.880287 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:33.880292 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:33.880753 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:33.880930 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:33.880939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:33.881978 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:33.881983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:33.882904 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:33.882915 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:33.883658 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:33.883664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:33.883838 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:33.884443 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:33.884461 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:33.889002 (+ 4541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":287,"threads_started":1}
W20260926 20:40:33.889453 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:33.879333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:47373 (local address 127.23.44.254:33723)
0926 20:40:33.879584 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:33.879592 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:33.879874 (+ 282us) server_negotiation.cc:407] Connection header received
0926 20:40:33.880070 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:33.880081 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:33.880154 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:33.880258 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:33.881081 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:33.881830 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:33.883051 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:33.883492 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:33.883632 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:33.889240 (+ 5608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:47373: BlockingRecv error: recv got EOF from 127.23.44.194:47373 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Sep 26 20:40:34 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:34 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:34.212708 545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:34.203002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:41295)
0926 20:40:34.203583 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:34.203598 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:34.203788 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:34.204156 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:34.204160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:34.204663 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:34.204843 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:34.204852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:34.206017 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:34.206022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:34.207040 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:34.207052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:34.207799 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:34.207804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:34.207986 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:34.208619 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:34.208638 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:34.212328 (+ 3690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":319,"threads_started":1}
W20260926 20:40:34.212801 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:34.203222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:41295 (local address 127.23.44.254:33723)
0926 20:40:34.203481 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:34.203488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:34.203692 (+ 204us) server_negotiation.cc:407] Connection header received
0926 20:40:34.203924 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:34.203930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:34.204009 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:34.204120 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:34.204986 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:34.205850 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:34.207179 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:34.207648 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:34.207788 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:34.212534 (+ 4746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:41295: BlockingRecv error: recv got EOF from 127.23.44.193:41295 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 26 20:40:34 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:34 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:34.600253 546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:34.590425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:37177)
0926 20:40:34.591006 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:34.591028 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:34.591202 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:34.591588 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:34.591593 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:34.592098 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:34.592338 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:34.592348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:34.593449 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:34.593455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:34.594380 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:34.594391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:34.595122 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:34.595128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:34.595309 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:34.595879 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:34.595903 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:34.599859 (+ 3956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":329,"threads_started":1}
W20260926 20:40:34.600363 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:34.590562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:37177 (local address 127.23.44.254:33723)
0926 20:40:34.590833 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:34.590840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:34.591137 (+ 297us) server_negotiation.cc:407] Connection header received
0926 20:40:34.591354 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:34.591360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:34.591441 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:34.591555 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:34.592484 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:34.593292 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:34.594546 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:34.594949 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:34.595087 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:34.600088 (+ 5001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:37177: BlockingRecv error: recv got EOF from 127.23.44.195:37177 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Sep 26 20:40:34 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:34 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:34.906335 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:34.890325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:45473)
0926 20:40:34.890953 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:34.890970 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:34.891102 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:34.891483 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:34.891486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:34.891845 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:34.892042 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:34.892051 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:34.893198 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:34.893202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:34.894179 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:34.894189 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:34.895031 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:34.895040 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:34.895177 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:34.901065 (+ 5888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:34.901097 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:34.905925 (+ 4828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":345,"threads_started":1}
W20260926 20:40:34.906579 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:34.890490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:45473 (local address 127.23.44.254:33723)
0926 20:40:34.890654 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:34.890659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:34.891056 (+ 397us) server_negotiation.cc:407] Connection header received
0926 20:40:34.891251 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:34.891255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:34.891321 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:34.891413 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:34.892194 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:34.893059 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:34.894372 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:34.894810 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:34.894943 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:34.906290 (+ 11347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:45473: BlockingRecv error: recv got EOF from 127.23.44.194:45473 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 20:40:35 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:35 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:35.223606 548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:35.213806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:40259)
0926 20:40:35.214422 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:35.214437 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:35.214550 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:35.214871 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:35.214875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:35.215235 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:35.215388 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:35.215396 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:35.216410 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:35.216414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:35.217503 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:35.217515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:35.218203 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:35.218208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:35.218345 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:35.219084 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:35.219129 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:35.223321 (+ 4192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":365,"threads_started":1}
W20260926 20:40:35.223779 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:35.213858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:40259 (local address 127.23.44.254:33723)
0926 20:40:35.214050 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:35.214059 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:35.214521 (+ 462us) server_negotiation.cc:407] Connection header received
0926 20:40:35.214640 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:35.214643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:35.214706 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:35.214797 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:35.215569 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:35.216282 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:35.217644 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:35.218047 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:35.218152 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:35.223532 (+ 5380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:40259: BlockingRecv error: recv got EOF from 127.23.44.193:40259 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 26 20:40:35 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:35 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:35.611826 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:35.601502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:50107)
0926 20:40:35.602101 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:35.602122 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:35.602297 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:35.602705 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:35.602710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:35.603141 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:35.603363 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:35.603372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:35.604457 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:35.604465 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:35.605567 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:35.605579 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:35.606380 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:35.606387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:35.606588 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:35.607325 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:35.607344 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:35.611593 (+ 4249us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":315,"threads_started":1}
W20260926 20:40:35.612131 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:35.601581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:50107 (local address 127.23.44.254:33723)
0926 20:40:35.601887 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:35.601894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:35.602223 (+ 329us) server_negotiation.cc:407] Connection header received
0926 20:40:35.602451 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:35.602458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:35.602539 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:35.602680 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:35.603537 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:35.604305 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:35.605735 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:35.606212 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:35.606353 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:35.611799 (+ 5446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:50107: BlockingRecv error: recv got EOF from 127.23.44.195:50107 (error 108)
Metrics: {"server-negotiator.queue_time_us":136}
Sep 26 20:40:35 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:35 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:35.918864 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:35.907535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:50105)
0926 20:40:35.908014 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:35.908031 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:35.908144 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:35.908478 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:35.908485 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:35.908932 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:35.909117 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:35.909126 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:35.910632 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:35.910637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:35.911771 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:35.911787 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:35.912721 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:35.912727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:35.912868 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:35.913736 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:35.913758 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:35.918391 (+ 4633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":275,"threads_started":1}
W20260926 20:40:35.918926 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:35.907582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:50105 (local address 127.23.44.254:33723)
0926 20:40:35.907815 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:35.907823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:35.908121 (+ 298us) server_negotiation.cc:407] Connection header received
0926 20:40:35.908240 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:35.908243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:35.908304 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:35.908383 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:35.909419 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:35.910475 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:35.911942 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:35.912421 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:35.912534 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:35.918655 (+ 6121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:50105: BlockingRecv error: recv got EOF from 127.23.44.194:50105 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 20:40:36 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:36 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:36.234187 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:36.224695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:33033)
0926 20:40:36.225280 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:36.225297 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:36.225412 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:36.225766 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:36.225769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:36.226167 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:36.226317 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:36.226325 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:36.227383 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:36.227387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:36.228330 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:36.228341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:36.229278 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:36.229283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:36.229425 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:36.230125 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:36.230143 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:36.233836 (+ 3693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":309,"threads_started":1}
W20260926 20:40:36.234283 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:36.224764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:33033 (local address 127.23.44.254:33723)
0926 20:40:36.224930 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:36.224935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:36.225380 (+ 445us) server_negotiation.cc:407] Connection header received
0926 20:40:36.225523 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:36.225528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:36.225595 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:36.225682 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:36.226452 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:36.227245 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:36.228662 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:36.229081 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:36.229192 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:36.234023 (+ 4831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:33033: BlockingRecv error: recv got EOF from 127.23.44.193:33033 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 26 20:40:36 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:36 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:36.623793 552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:36.613361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:42749)
0926 20:40:36.614193 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:36.614218 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:36.614383 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:36.614892 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:36.614898 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:36.615336 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:36.615592 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:36.615603 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:36.616747 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:36.616752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:36.617779 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:36.617792 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:36.618492 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:36.618496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:36.618624 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:36.619370 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:36.619388 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:36.623309 (+ 3921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":538,"threads_started":1}
W20260926 20:40:36.623919 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:36.613471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:42749 (local address 127.23.44.254:33723)
0926 20:40:36.613727 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:36.613734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:36.614305 (+ 571us) server_negotiation.cc:407] Connection header received
0926 20:40:36.614548 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:36.614554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:36.614638 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:36.614748 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:36.615756 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:36.616603 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:36.617932 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:36.618342 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:36.618444 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:36.623620 (+ 5176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:42749: BlockingRecv error: recv got EOF from 127.23.44.195:42749 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Sep 26 20:40:36 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:36 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:36.931146 553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:36.920132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:34941)
0926 20:40:36.920817 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:36.920841 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:36.920979 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:36.921432 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:36.921436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:36.921861 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:36.922038 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:36.922046 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:36.923647 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:36.923652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:36.925348 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:36.925364 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:36.926161 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:36.926168 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:36.926343 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:36.926955 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:36.926975 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:36.930896 (+ 3921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":408,"threads_started":1}
W20260926 20:40:36.931610 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:36.920497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:34941 (local address 127.23.44.254:33723)
0926 20:40:36.920851 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:36.920880 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:36.920913 (+ 33us) server_negotiation.cc:407] Connection header received
0926 20:40:36.921134 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:36.921146 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:36.921228 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:36.921343 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:36.922275 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:36.923431 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:36.925493 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:36.925950 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:36.926061 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:36.931275 (+ 5214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:34941: BlockingRecv error: recv got EOF from 127.23.44.194:34941 (error 108)
Metrics: {"server-negotiator.queue_time_us":196}
Sep 26 20:40:37 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:37 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:37.244961 554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:37.235158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:44255)
0926 20:40:37.235733 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:37.235749 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:37.235936 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:37.236318 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:37.236322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:37.236786 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:37.236963 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:37.236972 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:37.238086 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:37.238095 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:37.239136 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:37.239148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:37.239968 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:37.239974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:37.240190 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:37.240783 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:37.240802 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:37.244666 (+ 3864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":306,"threads_started":1}
W20260926 20:40:37.245159 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:37.235344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:44255 (local address 127.23.44.254:33723)
0926 20:40:37.235604 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:37.235611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:37.235846 (+ 235us) server_negotiation.cc:407] Connection header received
0926 20:40:37.236082 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:37.236092 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:37.236177 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:37.236289 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:37.237114 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:37.237817 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:37.239297 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:37.239791 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:37.239951 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:37.244856 (+ 4905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:44255: BlockingRecv error: recv got EOF from 127.23.44.193:44255 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Sep 26 20:40:37 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:37 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:37.634877 555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:37.625089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:52223)
0926 20:40:37.625680 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:37.625701 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:37.625814 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:37.626220 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:37.626226 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:37.626902 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:37.627096 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:37.627106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:37.628636 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:37.628642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:37.629694 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:37.629706 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:37.630406 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:37.630411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:37.630562 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:37.631191 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:37.631211 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:37.634630 (+ 3419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":334,"threads_started":1}
W20260926 20:40:37.635061 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:37.625154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:52223 (local address 127.23.44.254:33723)
0926 20:40:37.625368 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:37.625374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:37.625794 (+ 420us) server_negotiation.cc:407] Connection header received
0926 20:40:37.625934 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:37.625939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:37.626019 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:37.626141 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:37.627269 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:37.628269 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:37.629868 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:37.630258 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:37.630374 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:37.634825 (+ 4451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:52223: BlockingRecv error: recv got EOF from 127.23.44.195:52223 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 20:40:37 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:37 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:37.942857 556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:37.932408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:37049)
0926 20:40:37.932980 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:37.932998 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:37.933169 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:37.933525 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:37.933532 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:37.933961 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:37.934120 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:37.934129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:37.935173 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:37.935179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:37.936156 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:37.936170 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:37.937024 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:37.937030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:37.937200 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:37.937951 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:37.937968 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:37.942392 (+ 4424us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":322,"threads_started":1}
W20260926 20:40:37.942922 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:37.932513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:37049 (local address 127.23.44.254:33723)
0926 20:40:37.932823 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:37.932831 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:37.933095 (+ 264us) server_negotiation.cc:407] Connection header received
0926 20:40:37.933305 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:37.933315 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:37.933375 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:37.933484 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:37.934281 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:37.935030 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:37.936347 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:37.936854 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:37.936994 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:37.942657 (+ 5663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:37049: BlockingRecv error: recv got EOF from 127.23.44.194:37049 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Sep 26 20:40:38 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:38 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:38.256356 557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:38.246226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:57847)
0926 20:40:38.246883 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:38.246900 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:38.247032 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:38.247432 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:38.247435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:38.247846 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:38.248062 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:38.248072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:38.249341 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:38.249345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:38.250448 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:38.250460 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:38.251297 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:38.251303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:38.251470 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:38.252316 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:38.252336 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:38.256069 (+ 3733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":315,"threads_started":1}
W20260926 20:40:38.256996 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:38.246226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:57847 (local address 127.23.44.254:33723)
0926 20:40:38.246433 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:38.246438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:38.247027 (+ 589us) server_negotiation.cc:407] Connection header received
0926 20:40:38.247159 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:38.247164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:38.247253 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:38.247366 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:38.248252 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:38.249103 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:38.250604 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:38.251033 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:38.251153 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:38.256542 (+ 5389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:57847: BlockingRecv error: recv got EOF from 127.23.44.193:57847 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Sep 26 20:40:38 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:38 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:38.646152 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:38.636184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:51457)
0926 20:40:38.636781 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:38.636812 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:38.636989 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:38.637445 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:38.637451 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:38.637955 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:38.638120 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:38.638129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:38.639214 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:38.639218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:38.640172 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:38.640184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:38.641043 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:38.641047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:38.641175 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:38.641868 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:38.641885 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:38.645900 (+ 4015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":357,"threads_started":1}
W20260926 20:40:38.646553 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:38.636303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:51457 (local address 127.23.44.254:33723)
0926 20:40:38.636520 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:38.636528 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:38.636936 (+ 408us) server_negotiation.cc:407] Connection header received
0926 20:40:38.637153 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:38.637159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:38.637248 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:38.637368 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:38.638276 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:38.639057 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:38.640325 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:38.640891 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:38.640999 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:38.646309 (+ 5310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:51457: BlockingRecv error: recv got EOF from 127.23.44.195:51457 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 26 20:40:38 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:38 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:38.953433 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:38.944033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:58705)
0926 20:40:38.944720 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:38.944738 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:38.944862 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:38.945158 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:38.945161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:38.945586 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:38.945780 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:38.945789 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:38.946787 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:38.946792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:38.947884 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:38.947896 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:38.948760 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:38.948765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:38.948885 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:38.949530 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:38.949548 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:38.953164 (+ 3616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":381,"threads_started":1}
W20260926 20:40:38.953858 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:38.944118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:58705 (local address 127.23.44.254:33723)
0926 20:40:38.944373 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:38.944383 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:38.944820 (+ 437us) server_negotiation.cc:407] Connection header received
0926 20:40:38.944958 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:38.944962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:38.945031 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:38.945114 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:38.945904 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:38.946636 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:38.948044 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:38.948500 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:38.948647 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:38.953591 (+ 4944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:58705: BlockingRecv error: recv got EOF from 127.23.44.194:58705 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 26 20:40:39 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:39 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:39.268610 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:39.257523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:44159)
0926 20:40:39.258165 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:39.258181 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:39.258352 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:39.258699 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:39.258703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:39.259133 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:39.259295 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:39.259304 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:39.260332 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:39.260338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:39.261781 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:39.261801 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:39.262696 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:39.262702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:39.263055 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:39.264015 (+ 960us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:39.264036 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:39.268004 (+ 3968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":349,"threads_started":1}
W20260926 20:40:39.268978 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:39.257696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:44159 (local address 127.23.44.254:33723)
0926 20:40:39.257961 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:39.257970 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:39.258280 (+ 310us) server_negotiation.cc:407] Connection header received
0926 20:40:39.258492 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:39.258497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:39.258561 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:39.258669 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:39.259448 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:39.260184 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:39.261939 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:39.262502 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:39.262646 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:39.268457 (+ 5811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:44159: BlockingRecv error: recv got EOF from 127.23.44.193:44159 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Sep 26 20:40:39 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:39 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:39.657044 561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:39.647149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:54811)
0926 20:40:39.647820 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:39.647841 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:39.648019 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:39.648365 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:39.648369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:39.648827 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:39.649003 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:39.649013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:39.650010 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:39.650016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:39.650961 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:39.650973 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:39.651725 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:39.651732 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:39.651909 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:39.652646 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:39.652668 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:39.656738 (+ 4070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":334,"threads_started":1}
W20260926 20:40:39.657272 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:39.647397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:54811 (local address 127.23.44.254:33723)
0926 20:40:39.647663 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:39.647672 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:39.647940 (+ 268us) server_negotiation.cc:407] Connection header received
0926 20:40:39.648149 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:39.648156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:39.648227 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:39.648336 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:39.649155 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:39.649860 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:39.651108 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:39.651561 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:39.651692 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:39.656961 (+ 5269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:54811: BlockingRecv error: recv got EOF from 127.23.44.195:54811 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Sep 26 20:40:39 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:39 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:39.970250 562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:39.954631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:60045)
0926 20:40:39.955404 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:39.955420 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:39.955544 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:39.955836 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:39.955839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:39.956219 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:39.956388 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:39.956396 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:39.958098 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:39.958103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:39.959082 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:39.959094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:39.959948 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:39.959956 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:39.960144 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:39.966464 (+ 6320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:39.966485 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:39.969992 (+ 3507us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":509,"threads_started":1}
W20260926 20:40:39.970706 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:39.954712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:60045 (local address 127.23.44.254:33723)
0926 20:40:39.954947 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:39.954957 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:39.955527 (+ 570us) server_negotiation.cc:407] Connection header received
0926 20:40:39.955636 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:39.955639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:39.955704 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:39.955794 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:39.956525 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:39.957937 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:39.959233 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:39.959670 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:39.959819 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:39.970486 (+ 10667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:60045: BlockingRecv error: recv got EOF from 127.23.44.194:60045 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 26 20:40:40 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:40 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:40.280238 563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:40.269810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:53153)
0926 20:40:40.270294 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:40.270310 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:40.270479 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:40.270836 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:40.270841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:40.271275 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:40.271416 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:40.271424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:40.272454 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:40.272460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:40.273438 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:40.273449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:40.274156 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:40.274162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:40.274339 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:40.275139 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:40.275167 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:40.279826 (+ 4659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":276,"threads_started":1}
W20260926 20:40:40.280429 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:40.269834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:53153 (local address 127.23.44.254:33723)
0926 20:40:40.270035 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:40.270044 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:40.270400 (+ 356us) server_negotiation.cc:407] Connection header received
0926 20:40:40.270622 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:40.270628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:40.270701 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:40.270807 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:40.271558 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:40.272311 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:40.273584 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:40.274004 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:40.274141 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:40.280133 (+ 5992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:53153: BlockingRecv error: recv got EOF from 127.23.44.193:53153 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 26 20:40:40 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:40 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:40.669265 564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:40.658082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:60341)
0926 20:40:40.658596 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:40.658617 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:40.658745 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:40.659097 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:40.659103 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:40.659579 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:40.659743 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:40.659751 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:40.661090 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:40.661095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:40.662221 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:40.662236 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:40.663573 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:40.663584 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:40.663812 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:40.664725 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:40.664747 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:40.668981 (+ 4234us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":278,"threads_started":1}
W20260926 20:40:40.669754 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:40.658308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:60341 (local address 127.23.44.254:33723)
0926 20:40:40.658528 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:40.658537 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:40.658738 (+ 201us) server_negotiation.cc:407] Connection header received
0926 20:40:40.658853 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:40.658857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:40.658930 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:40.659019 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:40.660008 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:40.660905 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:40.662452 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:40.663249 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:40.663385 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:40.669381 (+ 5996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:60341: BlockingRecv error: recv got EOF from 127.23.44.195:60341 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Sep 26 20:40:40 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:40 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:40.981498 565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:40.971341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:38715)
0926 20:40:40.971869 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:40.971892 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:40.972032 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:40.972417 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:40.972422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:40.973285 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:40.973521 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:40.973535 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:40.974802 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:40.974809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:40.975798 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:40.975810 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:40.976618 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:40.976624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:40.976819 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:40.977468 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:40.977486 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:40.981154 (+ 3668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":261,"threads_started":1}
W20260926 20:40:40.981606 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:40.971396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:38715 (local address 127.23.44.254:33723)
0926 20:40:40.971646 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:40.971654 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:40.971966 (+ 312us) server_negotiation.cc:407] Connection header received
0926 20:40:40.972180 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:40.972186 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:40.972261 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:40.972375 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:40.973890 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:40.974645 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:40.975951 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:40.976384 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:40.976540 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:40.981349 (+ 4809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:38715: BlockingRecv error: recv got EOF from 127.23.44.194:38715 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 20:40:41 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:41 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:41.291891 566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:41.281691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:35223)
0926 20:40:41.282227 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:41.282244 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:41.282358 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:41.282719 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:41.282723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:41.283107 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:41.283260 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:41.283268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:41.284749 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:41.284753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:41.285798 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:41.285810 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:41.286643 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:41.286656 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:41.286770 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:41.287578 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:41.287633 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:41.291645 (+ 4012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":301,"threads_started":1}
W20260926 20:40:41.292168 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:41.281765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:35223 (local address 127.23.44.254:33723)
0926 20:40:41.281967 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:41.281972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:41.282352 (+ 380us) server_negotiation.cc:407] Connection header received
0926 20:40:41.282471 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:41.282476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:41.282544 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:41.282649 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:41.283414 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:41.284388 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:41.285976 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:41.286470 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:41.286661 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:41.291859 (+ 5198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:35223: BlockingRecv error: recv got EOF from 127.23.44.193:35223 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 26 20:40:41 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:41 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:41.680760 567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:41.670503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:55943)
0926 20:40:41.671053 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:41.671102 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:41.671252 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:41.671707 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:41.671711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:41.672124 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:41.672315 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:41.672327 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:41.673586 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:41.673591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:41.674543 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:41.674554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:41.675344 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:41.675349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:41.675492 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:41.676120 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:41.676139 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:41.680368 (+ 4229us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":293,"threads_started":1}
W20260926 20:40:41.680912 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:41.670805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:55943 (local address 127.23.44.254:33723)
0926 20:40:41.671048 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:41.671057 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:41.671299 (+ 242us) server_negotiation.cc:407] Connection header received
0926 20:40:41.671403 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:41.671409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:41.671525 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:41.671624 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:41.672544 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:41.673432 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:41.674712 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:41.675175 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:41.675292 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:41.680614 (+ 5322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:55943: BlockingRecv error: recv got EOF from 127.23.44.195:55943 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Sep 26 20:40:41 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:41 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 20:40:41.994462 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:41.982870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:60277 (local address 127.23.44.254:33723)
0926 20:40:41.983064 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:41.983071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:41.983549 (+ 478us) server_negotiation.cc:407] Connection header received
0926 20:40:41.983656 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:41.983662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:41.983888 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:41.984009 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:41.984863 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:41.985989 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:41.987683 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:41.988179 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:41.988325 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:41.994273 (+ 5948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:60277: BlockingRecv error: recv got EOF from 127.23.44.194:60277 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 20:40:41.994583 568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:41.982778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:60277)
0926 20:40:41.983376 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:41.983393 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:41.983517 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:41.984081 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:41.984085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:41.984446 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:41.984712 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:41.984723 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:41.986221 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:41.986227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:41.987475 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:41.987490 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:41.988394 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:41.988399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:41.988540 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:41.989462 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:41.989491 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:41.994035 (+ 4544us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":337,"threads_started":1}
Sep 26 20:40:42 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:42 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:42.302477 569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:42.292904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:47311)
0926 20:40:42.293433 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:42.293453 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:42.293607 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:42.293947 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:42.293951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:42.294414 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:42.294590 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:42.294598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:42.295650 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:42.295657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:42.296704 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:42.296716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:42.297413 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:42.297419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:42.297604 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:42.298231 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:42.298255 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:42.302243 (+ 3988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":297,"threads_started":1}
W20260926 20:40:42.302798 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:42.293227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:47311 (local address 127.23.44.254:33723)
0926 20:40:42.293433 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:42.293441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:42.293517 (+ 76us) server_negotiation.cc:407] Connection header received
0926 20:40:42.293739 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:42.293744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:42.293815 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:42.293920 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:42.294733 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:42.295515 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:42.296846 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:42.297257 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:42.297398 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:42.302520 (+ 5122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:47311: BlockingRecv error: recv got EOF from 127.23.44.193:47311 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 26 20:40:42 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:42 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:42.693235 570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:42.681896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:50087)
0926 20:40:42.682667 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:42.682692 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:42.682907 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:42.683483 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:42.683491 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:42.683968 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:42.684213 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:42.684225 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:42.685532 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:42.685540 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:42.686652 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:42.686665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:42.687479 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:42.687486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:42.687672 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:42.688490 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:42.688515 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:42.692934 (+ 4419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":434,"threads_started":1}
W20260926 20:40:42.693406 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:42.682061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:50087 (local address 127.23.44.254:33723)
0926 20:40:42.682465 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:42.682473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:42.682809 (+ 336us) server_negotiation.cc:407] Connection header received
0926 20:40:42.683131 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:42.683136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:42.683231 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:42.683340 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:42.684385 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:42.685381 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:42.686832 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:42.687294 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:42.687452 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:42.693151 (+ 5699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:50087: BlockingRecv error: recv got EOF from 127.23.44.195:50087 (error 108)
Metrics: {"server-negotiator.queue_time_us":194}
Sep 26 20:40:42 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:43 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:43.006450 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:42.995715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:45673)
0926 20:40:42.996169 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:42.996188 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:42.996315 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:42.996794 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:42.996798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:42.997199 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:42.997396 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:42.997404 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:42.998576 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:42.998580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:42.999522 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:42.999539 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:43.000871 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:43.000881 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:43.001128 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:43.002091 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:43.002122 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:43.006123 (+ 4001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":247,"threads_started":1}
W20260926 20:40:43.006649 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:42.995865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:45673 (local address 127.23.44.254:33723)
0926 20:40:42.996126 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:42.996134 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:42.996366 (+ 232us) server_negotiation.cc:407] Connection header received
0926 20:40:42.996471 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:42.996475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:42.996606 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:42.996734 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:42.997610 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:42.998416 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:42.999691 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:43.000442 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:43.000628 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:43.006365 (+ 5737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:45673: BlockingRecv error: recv got EOF from 127.23.44.194:45673 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Sep 26 20:40:43 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:43 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:43.316466 572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:43.303578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:45963)
0926 20:40:43.304142 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:43.304163 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:43.304343 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:43.304756 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:43.304762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:43.305747 (+ 985us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:43.306018 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:43.306031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:43.307525 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:43.307531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:43.308936 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:43.308951 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:43.310290 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:43.310299 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:43.310538 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:43.311249 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:43.311281 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:43.316027 (+ 4746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":258,"threads_started":1}
W20260926 20:40:43.316684 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:43.303956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:45963 (local address 127.23.44.254:33723)
0926 20:40:43.304192 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:43.304199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:43.304230 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:40:43.304478 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:43.304484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:43.304617 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:43.304718 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:43.306228 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:43.307372 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:43.309109 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:43.309827 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:43.309948 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:43.316304 (+ 6356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:45963: BlockingRecv error: recv got EOF from 127.23.44.193:45963 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Sep 26 20:40:43 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:43 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:43.706030 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:43.694359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:40585)
0926 20:40:43.694888 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:43.694912 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:43.695072 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:43.695552 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:43.695558 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:43.696165 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:43.696374 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:43.696384 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:43.697779 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:43.697784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:43.699143 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:43.699157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:43.699965 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:43.699972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:43.700175 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:43.700853 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:43.700872 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:43.705724 (+ 4852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":294,"threads_started":1}
W20260926 20:40:43.706157 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:43.694596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:40585 (local address 127.23.44.254:33723)
0926 20:40:43.694882 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:43.694888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:43.695029 (+ 141us) server_negotiation.cc:407] Connection header received
0926 20:40:43.695186 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:43.695190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:43.695257 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:43.695349 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:43.696609 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:43.697598 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:43.699305 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:43.699788 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:43.699948 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:43.705922 (+ 5974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:40585: BlockingRecv error: recv got EOF from 127.23.44.195:40585 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Sep 26 20:40:44 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:44 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:44.017419 574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:44.007547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:57639)
0926 20:40:44.008270 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:44.008291 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:44.008486 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:44.009039 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:44.009043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:44.009487 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:44.009667 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:44.009675 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:44.010885 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:44.010889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:44.011860 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:44.011872 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:44.012742 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:44.012747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:44.012875 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:44.013591 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:44.013609 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:44.017166 (+ 3557us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":399,"threads_started":1}
W20260926 20:40:44.017823 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:44.007627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:57639 (local address 127.23.44.254:33723)
0926 20:40:44.007934 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:44.007946 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:44.008469 (+ 523us) server_negotiation.cc:407] Connection header received
0926 20:40:44.008741 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:44.008746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:44.008825 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:44.008943 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:44.009901 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:44.010729 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:44.012005 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:44.012473 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:44.012607 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:44.017491 (+ 4884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:57639: BlockingRecv error: recv got EOF from 127.23.44.194:57639 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Sep 26 20:40:44 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:44 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:44.328871 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:44.317929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:36801)
0926 20:40:44.318546 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:44.318566 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:44.318796 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:44.319181 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:44.319187 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:44.319666 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:44.319815 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:44.319824 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:44.321078 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:44.321089 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:44.322244 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:44.322257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:44.323032 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:44.323038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:44.323214 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:44.324034 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:44.324061 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:44.328600 (+ 4539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":346,"threads_started":1}
W20260926 20:40:44.329079 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:44.318024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:36801 (local address 127.23.44.254:33723)
0926 20:40:44.318329 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:44.318339 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:44.318675 (+ 336us) server_negotiation.cc:407] Connection header received
0926 20:40:44.318942 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:44.318949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:44.319026 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:44.319148 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:44.319975 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:44.320925 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:44.322410 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:44.322850 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:44.323014 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:44.328841 (+ 5827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:36801: BlockingRecv error: recv got EOF from 127.23.44.193:36801 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Sep 26 20:40:44 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:44 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:44.717247 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:44.707240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:42469)
0926 20:40:44.708078 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:44.708100 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:44.708212 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:44.708695 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:44.708700 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:44.709232 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:44.709456 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:44.709466 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:44.710638 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:44.710643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:44.711643 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:44.711656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:44.712410 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:44.712415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:44.712547 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:44.713295 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:44.713313 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:44.716982 (+ 3669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":554,"threads_started":1}
W20260926 20:40:44.717411 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:44.707204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:42469 (local address 127.23.44.254:33723)
0926 20:40:44.707465 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:44.707476 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:44.708182 (+ 706us) server_negotiation.cc:407] Connection header received
0926 20:40:44.708315 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:44.708318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:44.708380 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:44.708468 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:44.709681 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:44.710463 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:44.711813 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:44.712237 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:44.712340 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:44.717198 (+ 4858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:42469: BlockingRecv error: recv got EOF from 127.23.44.195:42469 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 26 20:40:45 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:45 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:45.031658 577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:45.018520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:32945)
0926 20:40:45.019057 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:45.019074 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:45.019190 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:45.019505 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:45.019508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:45.020038 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:45.020248 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:45.020258 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:45.021353 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:45.021357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:45.022271 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:45.022282 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:45.023048 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:45.023051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:45.023183 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:45.027722 (+ 4539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:45.027743 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:45.031427 (+ 3684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":300,"threads_started":1}
W20260926 20:40:45.031935 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:45.018730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:32945 (local address 127.23.44.254:33723)
0926 20:40:45.018959 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:45.018968 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:45.019186 (+ 218us) server_negotiation.cc:407] Connection header received
0926 20:40:45.019299 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:45.019303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:45.019372 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:45.019474 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:45.020418 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:45.021217 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:45.022404 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:45.022896 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:45.023031 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:45.031673 (+ 8642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:32945: BlockingRecv error: recv got EOF from 127.23.44.194:32945 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 26 20:40:45 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:45 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:45.339756 578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:45.330257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:52759)
0926 20:40:45.330766 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:45.330782 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:45.330886 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:45.331302 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:45.331306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:45.331656 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:45.331840 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:45.331848 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:45.333366 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:45.333370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:45.334328 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:45.334340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:45.335093 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:45.335098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:45.335238 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:45.335871 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:45.335889 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:45.339482 (+ 3593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":262,"threads_started":1}
W20260926 20:40:45.339933 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:45.330336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:52759 (local address 127.23.44.254:33723)
0926 20:40:45.330554 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:45.330559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:45.330865 (+ 306us) server_negotiation.cc:407] Connection header received
0926 20:40:45.331020 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:45.331025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:45.331109 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:45.331199 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:45.332110 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:45.333204 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:45.334488 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:45.334927 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:45.335045 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:45.339713 (+ 4668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:52759: BlockingRecv error: recv got EOF from 127.23.44.193:52759 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 26 20:40:45 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:45 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.195@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:45.728420 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:45.718422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.195:44303)
0926 20:40:45.718980 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:45.719004 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:45.719118 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:45.719506 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:45.719510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:45.720009 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:45.720178 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:45.720188 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:45.721385 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:45.721390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:45.722467 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:45.722479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:45.723741 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:45.723748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:45.723931 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:45.724650 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:45.724668 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:45.728152 (+ 3484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":328,"threads_started":1}
W20260926 20:40:45.729013 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:45.718554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:44303 (local address 127.23.44.254:33723)
0926 20:40:45.718842 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:45.718861 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:45.719112 (+ 251us) server_negotiation.cc:407] Connection header received
0926 20:40:45.719211 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:45.719214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:45.719290 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:45.719378 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:45.720339 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:45.721201 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:45.722764 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:45.723568 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:45.723725 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:45.728711 (+ 4986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.195:44303: BlockingRecv error: recv got EOF from 127.23.44.195:44303 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Sep 26 20:40:46 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:46 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.194@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:46.042258 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:46.032912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.194:57601)
0926 20:40:46.033436 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:46.033460 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:46.033576 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:46.033913 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:46.033916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:46.034271 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:46.034449 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:46.034457 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.035529 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.035533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:46.036465 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:46.036476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.037583 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.037588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:46.037728 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:46.038367 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:46.038384 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:46.041935 (+ 3551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":268,"threads_started":1}
W20260926 20:40:46.042641 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:46.033003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:57601 (local address 127.23.44.254:33723)
0926 20:40:46.033310 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:46.033316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:46.033588 (+ 272us) server_negotiation.cc:407] Connection header received
0926 20:40:46.033675 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:46.033679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:46.033748 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:46.033835 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:46.034603 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.035381 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.036693 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.037410 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.037550 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:46.042235 (+ 4685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.194:57601: BlockingRecv error: recv got EOF from 127.23.44.194:57601 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 20:40:46 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 20:40:46 dist-test-slave-4d5t krb5kdc[32000](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.44.193@KRBTEST.COM for oryx/127.23.44.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 20:40:46.351111 581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:46.340898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33723 (local address 127.23.44.193:42377)
0926 20:40:46.341526 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:46.341542 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:46.341720 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:46.342322 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:46.342328 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:46.342865 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:46.343056 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:46.343066 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.344284 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.344290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:46.345442 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:46.345454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.346305 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.346311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:46.346499 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:46.347156 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:46.347174 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:46.350871 (+ 3697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.44.254:33723: Server oryx/127.23.44.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":337,"threads_started":1}
W20260926 20:40:46.351384 32738 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:46.341065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:42377 (local address 127.23.44.254:33723)
0926 20:40:46.341323 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:46.341332 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:46.341650 (+ 318us) server_negotiation.cc:407] Connection header received
0926 20:40:46.341862 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:46.341941 (+ 79us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:46.342066 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:46.342256 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:46.343343 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.344122 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.345592 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.346116 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.346277 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:46.351118 (+ 4841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.44.193:42377: BlockingRecv error: recv got EOF from 127.23.44.193:42377 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260926 20:40:46.406379 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 32589
I20260926 20:40:46.420749 583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:46.407693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42302 (local address 127.23.44.193:38829)
0926 20:40:46.408302 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:46.408312 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:46.408350 (+ 38us) server_negotiation.cc:407] Connection header received
0926 20:40:46.408429 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:46.408437 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:46.408762 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:46.408944 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:46.410329 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.411414 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.412808 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.413700 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.413841 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:46.416798 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:46.416808 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:46.416823 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:46.416877 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:46.419980 (+ 3103us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.23.44.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 20:40:46.420376 (+ 396us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42302: Request ticket server kudu/127.23.44.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":547,"threads_started":1}
W20260926 20:40:46.420790 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 32589: Not authorized: Client connection negotiation failed: client connection to 127.23.44.193:38829: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.23.44.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 20:40:46.420856 583 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42302: Request ticket server kudu/127.23.44.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 20:40:46.421005 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 32589
I20260926 20:40:46.439905 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 32740
I20260926 20:40:46.456537 584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:46.441182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.23.44.194:45617)
0926 20:40:46.441841 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:46.441849 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:46.441880 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:40:46.441945 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:46.441952 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:46.442197 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:46.442435 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:46.443888 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.445461 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.446905 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.447911 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.448059 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:46.451599 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:46.451616 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:46.451643 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:46.451726 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:46.455787 (+ 4061us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.23.44.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 20:40:46.456213 (+ 426us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41960: Request ticket server kudu/127.23.44.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":344,"threads_started":1}
W20260926 20:40:46.456656 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 32740: Not authorized: Client connection negotiation failed: client connection to 127.23.44.194:45617: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.23.44.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 20:40:46.456764 584 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41960: Request ticket server kudu/127.23.44.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 20:40:46.456848 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 32740
I20260926 20:40:46.471487 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 405
I20260926 20:40:46.485582 585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:46.472654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54482 (local address 127.23.44.195:42637)
0926 20:40:46.473265 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:46.473273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:46.473302 (+ 29us) server_negotiation.cc:407] Connection header received
0926 20:40:46.473374 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:46.473382 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:46.473618 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:46.473816 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:46.475985 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.477485 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.478693 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:46.479264 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:46.479378 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:46.481877 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:46.481887 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:46.481902 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:46.481941 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:46.484939 (+ 2998us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.23.44.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 20:40:46.485263 (+ 324us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54482: Request ticket server kudu/127.23.44.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":302,"threads_started":1}
W20260926 20:40:46.485700 585 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54482: Request ticket server kudu/127.23.44.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 20:40:46.485757 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 405: Not authorized: Client connection negotiation failed: client connection to 127.23.44.195:42637: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.23.44.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 20:40:46.485965 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 405
I20260926 20:40:46.503563 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 32520
I20260926 20:40:46.505198 32582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:46.624784 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 32520
2026-09-26T20:40:46Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21979 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T20:40:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:40:46Z Disabled control of system clock
I20260926 20:40:46.708854 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:39681
--webserver_interface=127.23.44.254
--webserver_port=0
--builtin_ntp_servers=127.23.44.212:45717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:39681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 20:40:47.063140 590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:47.063458 590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:47.063575 590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:47.071355 590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:40:47.071476 590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:47.071543 590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:47.071635 590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:40:47.071674 590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:40:47.081233 590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:39681
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:39681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:47.082705 590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:47.084538 590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:47.096817 595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:47.096779 598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:47.096740 596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:47.098042 590 server_base.cc:1034] running on GCE node
I20260926 20:40:47.098948 590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:47.100713 590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:47.101956 590 hybrid_clock.cc:648] HybridClock initialized: now 1790455247101895 us; error 57 us; skew 500 ppm
I20260926 20:40:47.102363 590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:40:47.104912 590 webserver.cc:460] Webserver started at http://127.23.44.254:36457/ using document root <none> and password file <none>
I20260926 20:40:47.105643 590 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:47.105763 590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:47.106110 590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:47.109150 590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "858bbd9317c247108462e5da6f70b47a"
format_stamp: "Formatted at 2026-09-26 20:40:47 on dist-test-slave-4d5t"
server_key: "688a859ec76952d878309c6c8086d31c"
server_key_iv: "780c4fa3367934bd0611d66bf6e54da6"
server_key_version: "encryptionkey@0"
I20260926 20:40:47.110190 590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "858bbd9317c247108462e5da6f70b47a"
format_stamp: "Formatted at 2026-09-26 20:40:47 on dist-test-slave-4d5t"
server_key: "688a859ec76952d878309c6c8086d31c"
server_key_iv: "780c4fa3367934bd0611d66bf6e54da6"
server_key_version: "encryptionkey@0"
I20260926 20:40:47.115949 590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260926 20:40:47.120491 604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:47.122292 590 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:40:47.122516 590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "858bbd9317c247108462e5da6f70b47a"
format_stamp: "Formatted at 2026-09-26 20:40:47 on dist-test-slave-4d5t"
server_key: "688a859ec76952d878309c6c8086d31c"
server_key_iv: "780c4fa3367934bd0611d66bf6e54da6"
server_key_version: "encryptionkey@0"
I20260926 20:40:47.122761 590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:47.138655 590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:47.139614 590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:47.139954 590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:47.161105 590 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:39681
I20260926 20:40:47.161124 655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:39681 every 8 connection(s)
I20260926 20:40:47.162595 590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:40:47.163532 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 590
I20260926 20:40:47.163740 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:40:47.165009 23731 external_mini_cluster.cc:1428] Setting key 42a0afb4ed4378f2521ab646aaacf936
I20260926 20:40:47.167016 656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:47.178912 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:47.167139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56508 (local address 127.23.44.254:39681)
0926 20:40:47.168223 (+ 1084us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:47.168235 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:47.168285 (+ 50us) server_negotiation.cc:407] Connection header received
0926 20:40:47.169938 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:47.169960 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:47.170423 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:47.170849 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:47.171449 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:47.172858 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:47.174235 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:47.174909 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:47.175113 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:47.176341 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:47.176349 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:47.176364 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:47.177123 (+ 759us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:47.177206 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:47.177230 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:47.177409 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:47.177652 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 20:40:47.177971 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":349,"threads_started":1}
I20260926 20:40:47.179561 656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a: Bootstrap starting.
I20260926 20:40:47.182900 656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:47.184031 656 log.cc:824] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:47.186762 656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a: No bootstrap required, opened a new log
I20260926 20:40:47.194304 656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858bbd9317c247108462e5da6f70b47a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39681 } }
I20260926 20:40:47.194725 656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:47.194831 656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 858bbd9317c247108462e5da6f70b47a, State: Initialized, Role: FOLLOWER
I20260926 20:40:47.195446 656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858bbd9317c247108462e5da6f70b47a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39681 } }
I20260926 20:40:47.195696 656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:40:47.195837 656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:40:47.196048 656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:47.198300 656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858bbd9317c247108462e5da6f70b47a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39681 } }
I20260926 20:40:47.198804 656 leader_election.cc:302] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 858bbd9317c247108462e5da6f70b47a; no voters:
I20260926 20:40:47.199453 656 leader_election.cc:288] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:40:47.199738 661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:47.200511 661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [term 1 LEADER]: Becoming Leader. State: Replica: 858bbd9317c247108462e5da6f70b47a, State: Running, Role: LEADER
I20260926 20:40:47.201143 661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858bbd9317c247108462e5da6f70b47a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39681 } }
I20260926 20:40:47.201547 656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:40:47.204212 663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 858bbd9317c247108462e5da6f70b47a. Latest consensus state: current_term: 1 leader_uuid: "858bbd9317c247108462e5da6f70b47a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858bbd9317c247108462e5da6f70b47a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39681 } } }
I20260926 20:40:47.204525 662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "858bbd9317c247108462e5da6f70b47a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "858bbd9317c247108462e5da6f70b47a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39681 } } }
I20260926 20:40:47.204850 663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:47.204919 662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:47.216063 669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:40:47.221571 669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:40:47.231595 669 catalog_manager.cc:1347] Generated new cluster ID: 26cb08725a754e9396555ba487d870c4
I20260926 20:40:47.231717 669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:40:47.256717 669 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:40:47.258100 669 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:40:47.265496 669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 858bbd9317c247108462e5da6f70b47a: Generated new TSK 0
I20260926 20:40:47.266378 669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:40:47.282863 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39681
--builtin_ntp_servers=127.23.44.212:45717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 20:40:47.633790 680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:47.634095 680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:47.634228 680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:47.642726 680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:47.642904 680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:47.643172 680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:40:47.654434 680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:47.655848 680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:47.657667 680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:47.671130 685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:47.671233 688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:47.671233 686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:47.671871 680 server_base.cc:1034] running on GCE node
I20260926 20:40:47.673125 680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:47.674536 680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:47.675824 680 hybrid_clock.cc:648] HybridClock initialized: now 1790455247675794 us; error 58 us; skew 500 ppm
I20260926 20:40:47.676234 680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:40:47.679273 680 webserver.cc:460] Webserver started at http://127.23.44.193:46589/ using document root <none> and password file <none>
I20260926 20:40:47.680070 680 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:47.680223 680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:47.680652 680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:47.683817 680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "1cb823dd0b6d4085b4501335e2306e52"
format_stamp: "Formatted at 2026-09-26 20:40:47 on dist-test-slave-4d5t"
server_key: "e57594be6858f50f499f8a7d31e96e54"
server_key_iv: "32b2330647d2c52003a54ea5b3383db7"
server_key_version: "encryptionkey@0"
I20260926 20:40:47.684800 680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "1cb823dd0b6d4085b4501335e2306e52"
format_stamp: "Formatted at 2026-09-26 20:40:47 on dist-test-slave-4d5t"
server_key: "e57594be6858f50f499f8a7d31e96e54"
server_key_iv: "32b2330647d2c52003a54ea5b3383db7"
server_key_version: "encryptionkey@0"
I20260926 20:40:47.690941 680 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 20:40:47.695886 694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:47.697568 680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 20:40:47.697782 680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "1cb823dd0b6d4085b4501335e2306e52"
format_stamp: "Formatted at 2026-09-26 20:40:47 on dist-test-slave-4d5t"
server_key: "e57594be6858f50f499f8a7d31e96e54"
server_key_iv: "32b2330647d2c52003a54ea5b3383db7"
server_key_version: "encryptionkey@0"
I20260926 20:40:47.698058 680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:47.729049 680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:47.729959 680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:47.730274 680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:47.731340 680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:47.732908 680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:47.733028 680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:47.733168 680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:47.733261 680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:47.771847 680 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:43515
I20260926 20:40:47.771903 806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:43515 every 8 connection(s)
I20260926 20:40:47.773653 680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:40:47.782982 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 680
I20260926 20:40:47.783191 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:40:47.783885 23731 external_mini_cluster.cc:1428] Setting key cf5fbe944272df2563b5a0571bc3447e
I20260926 20:40:47.786898 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39681
--builtin_ntp_servers=127.23.44.212:45717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 20:40:47.787919 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:47.776438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:48427 (local address 127.23.44.254:39681)
0926 20:40:47.776943 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:47.776948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:47.778365 (+ 1417us) server_negotiation.cc:407] Connection header received
0926 20:40:47.780300 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:47.780305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:47.780411 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:47.780546 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:47.781758 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:47.782700 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:47.784545 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:47.785107 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:47.785252 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:47.786790 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:47.786796 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:47.786801 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:47.786824 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:47.786876 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:47.786908 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:47.787017 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:47.787306 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 20:40:47.787421 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":266,"threads_started":1}
I20260926 20:40:47.788705 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:47.777097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39681 (local address 127.23.44.193:48427)
0926 20:40:47.778143 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:47.778203 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:47.780003 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:47.780819 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:47.780828 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:47.780934 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:47.781573 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:47.781592 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:47.782907 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:47.782911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:47.784364 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:47.784376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:47.785366 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:47.785372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:47.785670 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:47.786455 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:40:47.786463 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:47.786504 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:40:47.786586 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:47.787048 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:47.787061 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:47.787072 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 20:40:47.787271 (+ 199us) client_negotiation.cc:240] Negotiation successful
0926 20:40:47.787575 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":318,"threads_started":1}
I20260926 20:40:47.790266 807 heartbeater.cc:347] Connected to a master server at 127.23.44.254:39681
I20260926 20:40:47.790608 807 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:47.791603 807 heartbeater.cc:511] Master 127.23.44.254:39681 requested a full tablet report, sending...
I20260926 20:40:47.793946 621 ts_manager.cc:194] Registered new tserver with Master: 1cb823dd0b6d4085b4501335e2306e52 (127.23.44.193:43515)
I20260926 20:40:47.795598 621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.44.193:48427
W20260926 20:40:48.129988 811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:48.130291 811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:48.130426 811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:48.138208 811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:48.138334 811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:48.138526 811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:40:48.148312 811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:48.149740 811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:48.151525 811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:48.165361 819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:48.165354 817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:48.166095 816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:48.166513 811 server_base.cc:1034] running on GCE node
I20260926 20:40:48.167253 811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:48.168416 811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:48.169651 811 hybrid_clock.cc:648] HybridClock initialized: now 1790455248169586 us; error 82 us; skew 500 ppm
I20260926 20:40:48.170003 811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:40:48.172849 811 webserver.cc:460] Webserver started at http://127.23.44.194:33733/ using document root <none> and password file <none>
I20260926 20:40:48.173820 811 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:48.173938 811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:48.174295 811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:48.178218 811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "a5e1ce06980c4217b591b08657aa94eb"
format_stamp: "Formatted at 2026-09-26 20:40:48 on dist-test-slave-4d5t"
server_key: "a0f09bfb4c666d663d2b08c45a011a34"
server_key_iv: "d30d1846db00b19d687c336e9c8fe7ac"
server_key_version: "encryptionkey@0"
I20260926 20:40:48.179278 811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "a5e1ce06980c4217b591b08657aa94eb"
format_stamp: "Formatted at 2026-09-26 20:40:48 on dist-test-slave-4d5t"
server_key: "a0f09bfb4c666d663d2b08c45a011a34"
server_key_iv: "d30d1846db00b19d687c336e9c8fe7ac"
server_key_version: "encryptionkey@0"
I20260926 20:40:48.185730 811 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260926 20:40:48.189720 825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:48.191313 811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260926 20:40:48.191537 811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "a5e1ce06980c4217b591b08657aa94eb"
format_stamp: "Formatted at 2026-09-26 20:40:48 on dist-test-slave-4d5t"
server_key: "a0f09bfb4c666d663d2b08c45a011a34"
server_key_iv: "d30d1846db00b19d687c336e9c8fe7ac"
server_key_version: "encryptionkey@0"
I20260926 20:40:48.191772 811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:48.216969 811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:48.217860 811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:48.218180 811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:48.219159 811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:48.220695 811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:48.220819 811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:48.220964 811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:48.221055 811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:48.257927 811 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:37635
I20260926 20:40:48.258018 937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:37635 every 8 connection(s)
I20260926 20:40:48.259246 811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:40:48.265921 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 811
I20260926 20:40:48.266109 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:40:48.266666 23731 external_mini_cluster.cc:1428] Setting key 8adab1d1664c474c170122ee702b301e
I20260926 20:40:48.269860 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39681
--builtin_ntp_servers=127.23.44.212:45717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 20:40:48.276757 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:48.262776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:40685 (local address 127.23.44.254:39681)
0926 20:40:48.263059 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:48.263065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:48.265114 (+ 2049us) server_negotiation.cc:407] Connection header received
0926 20:40:48.267801 (+ 2687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:48.267810 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:48.267917 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:48.268041 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:48.269492 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:48.270412 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:48.273187 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:48.273856 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:48.274003 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:48.275789 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:48.275797 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:48.275805 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:48.275831 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:48.275867 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:48.275883 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:48.275961 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:48.276238 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 20:40:48.276342 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260926 20:40:48.278813 940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:48.263459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39681 (local address 127.23.44.194:40685)
0926 20:40:48.264908 (+ 1449us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:48.264957 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:48.267407 (+ 2450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:48.268285 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:48.268295 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:48.268405 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:48.269304 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:48.269329 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:48.270582 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:48.270590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:48.272865 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:48.272883 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:48.274149 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:48.274158 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:48.274516 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:48.275385 (+ 869us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:40:48.275391 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:48.275448 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:40:48.275565 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:48.275986 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:48.275998 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:48.276009 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 20:40:48.276204 (+ 195us) client_negotiation.cc:240] Negotiation successful
0926 20:40:48.276933 (+ 729us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":446,"threads_started":1}
I20260926 20:40:48.281206 938 heartbeater.cc:347] Connected to a master server at 127.23.44.254:39681
I20260926 20:40:48.281602 938 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:48.282424 938 heartbeater.cc:511] Master 127.23.44.254:39681 requested a full tablet report, sending...
I20260926 20:40:48.283977 621 ts_manager.cc:194] Registered new tserver with Master: a5e1ce06980c4217b591b08657aa94eb (127.23.44.194:37635)
I20260926 20:40:48.284941 621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.44.194:40685
W20260926 20:40:48.615082 941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:48.615409 941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:48.615501 941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:48.624308 941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:48.624498 941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:48.624780 941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:40:48.635437 941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:48.636813 941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:48.638593 941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:48.654590 946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:48.654661 949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:48.654597 947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:48.655870 941 server_base.cc:1034] running on GCE node
I20260926 20:40:48.656642 941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:48.657721 941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:48.658979 941 hybrid_clock.cc:648] HybridClock initialized: now 1790455248658923 us; error 61 us; skew 500 ppm
I20260926 20:40:48.659327 941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:40:48.661834 941 webserver.cc:460] Webserver started at http://127.23.44.195:34095/ using document root <none> and password file <none>
I20260926 20:40:48.662582 941 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:48.662724 941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:48.663070 941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:48.666149 941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "5e44b86eb0314888b67427677955b975"
format_stamp: "Formatted at 2026-09-26 20:40:48 on dist-test-slave-4d5t"
server_key: "c7ecf7dce5a07f15bf818af98887be51"
server_key_iv: "d4d76ea1ace6d6bda35c1e14270d4d00"
server_key_version: "encryptionkey@0"
I20260926 20:40:48.667003 941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "5e44b86eb0314888b67427677955b975"
format_stamp: "Formatted at 2026-09-26 20:40:48 on dist-test-slave-4d5t"
server_key: "c7ecf7dce5a07f15bf818af98887be51"
server_key_iv: "d4d76ea1ace6d6bda35c1e14270d4d00"
server_key_version: "encryptionkey@0"
I20260926 20:40:48.673492 941 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 20:40:48.678524 955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:48.680377 941 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260926 20:40:48.680685 941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "5e44b86eb0314888b67427677955b975"
format_stamp: "Formatted at 2026-09-26 20:40:48 on dist-test-slave-4d5t"
server_key: "c7ecf7dce5a07f15bf818af98887be51"
server_key_iv: "d4d76ea1ace6d6bda35c1e14270d4d00"
server_key_version: "encryptionkey@0"
I20260926 20:40:48.681005 941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:48.709931 941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:48.710904 941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:48.711213 941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:48.712200 941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:48.713745 941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:48.713888 941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:48.714018 941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:48.714102 941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:48.753742 941 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:42357
I20260926 20:40:48.753789 1067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:42357 every 8 connection(s)
I20260926 20:40:48.755035 941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:40:48.764370 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 941
I20260926 20:40:48.764684 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:40:48.765300 23731 external_mini_cluster.cc:1428] Setting key edc6ddf6cf8a553f95aba0d3a2ad947b
I20260926 20:40:48.769863 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:48.757779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:59459 (local address 127.23.44.254:39681)
0926 20:40:48.757990 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:48.757995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:48.759369 (+ 1374us) server_negotiation.cc:407] Connection header received
0926 20:40:48.762413 (+ 3044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:48.762419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:48.762522 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:48.762634 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:48.763772 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:48.764624 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:48.766386 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:48.767149 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:48.767273 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:48.769007 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:48.769013 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:48.769018 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:48.769049 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:48.769088 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:48.769105 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:48.769203 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:48.769543 (+ 340us) server_negotiation.cc:299] Negotiation successful
0926 20:40:48.769626 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 20:40:48.770949 1070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:48.758265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39681 (local address 127.23.44.195:59459)
0926 20:40:48.759196 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:48.759231 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:48.762052 (+ 2821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:48.762887 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:48.762897 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:48.762999 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:48.763602 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:48.763618 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:48.764786 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:48.764792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:48.766184 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:48.766199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:48.767424 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:48.767433 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:48.767723 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:48.768631 (+ 908us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:40:48.768634 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:48.768676 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:40:48.768763 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:48.769240 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:48.769256 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:48.769265 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 20:40:48.769472 (+ 207us) client_negotiation.cc:240] Negotiation successful
0926 20:40:48.769786 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":287,"threads_started":1}
I20260926 20:40:48.772676 1068 heartbeater.cc:347] Connected to a master server at 127.23.44.254:39681
I20260926 20:40:48.772993 1068 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:48.773787 1068 heartbeater.cc:511] Master 127.23.44.254:39681 requested a full tablet report, sending...
I20260926 20:40:48.775436 621 ts_manager.cc:194] Registered new tserver with Master: 5e44b86eb0314888b67427677955b975 (127.23.44.195:42357)
I20260926 20:40:48.776490 621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.44.195:59459
I20260926 20:40:48.781781 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 20:40:48.788604 809 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:48.786877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.23.44.254:39681)
0926 20:40:48.787195 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:48.787201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:48.787660 (+ 459us) server_negotiation.cc:407] Connection header received
0926 20:40:48.787764 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:48.787768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:48.787834 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:48.787933 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:48.788292 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56510: BlockingRecv error: recv got EOF from 127.0.0.1:56510 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
I20260926 20:40:48.789505 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 680
I20260926 20:40:48.798358 1077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:48.790551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.23.44.193:43515)
0926 20:40:48.791117 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:48.791124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:48.791150 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:40:48.791205 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:48.791212 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:48.791434 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:48.791556 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:48.792063 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:48.793359 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:48.794740 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:48.795342 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:48.795460 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:48.796877 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:48.796886 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:48.796901 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:48.797487 (+ 586us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:48.797533 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:48.797553 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:48.797662 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:48.797824 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 20:40:48.797940 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":302,"threads_started":1}
I20260926 20:40:48.799623 807 heartbeater.cc:503] Master 127.23.44.254:39681 was elected leader, sending a full tablet report...
I20260926 20:40:48.799736 802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:48.882230 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 680
I20260926 20:40:48.901859 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 811
I20260926 20:40:48.911531 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:48.902882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56290 (local address 127.23.44.194:37635)
0926 20:40:48.903464 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:48.903471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:48.903500 (+ 29us) server_negotiation.cc:407] Connection header received
0926 20:40:48.903560 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:48.903567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:48.903785 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:48.904144 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:48.904492 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:48.905976 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:48.907420 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:48.908240 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:48.908391 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:48.909790 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:48.909844 (+ 54us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:48.909870 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:48.910586 (+ 716us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:48.910644 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:48.910666 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:48.910773 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:48.911005 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 20:40:48.911211 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":303,"threads_started":1}
I20260926 20:40:48.913000 933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:49.000145 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 811
I20260926 20:40:49.016649 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 941
I20260926 20:40:49.025631 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:49.017733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42482 (local address 127.23.44.195:42357)
0926 20:40:49.018366 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:49.018373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:49.018404 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:40:49.018461 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:49.018474 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:49.018710 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:49.018887 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:49.019483 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:49.020511 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:49.021849 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:49.022433 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:49.022578 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:49.024044 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:49.024053 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:49.024068 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:49.024746 (+ 678us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:49.024797 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:49.024818 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:49.024955 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:49.025138 (+ 183us) server_negotiation.cc:299] Negotiation successful
0926 20:40:49.025268 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":330,"threads_started":1}
I20260926 20:40:49.026980 1063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:49.116142 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 941
I20260926 20:40:49.132992 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 590
I20260926 20:40:49.134408 651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:49.238664 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 590
2026-09-26T20:40:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2608 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T20:40:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:40:49Z Disabled control of system clock
I20260926 20:40:49.317672 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:39593
--webserver_interface=127.23.44.254
--webserver_port=0
--builtin_ntp_servers=127.23.44.212:46227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:39593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 20:40:49.655072 1087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:49.655380 1087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:49.655540 1087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:49.663040 1087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:40:49.663162 1087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:49.663235 1087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:49.663347 1087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:40:49.663429 1087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:40:49.672986 1087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:46227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:39593
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:39593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:49.674306 1087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:49.676096 1087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:49.688704 1092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:49.688637 1095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:49.688681 1093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:49.689909 1087 server_base.cc:1034] running on GCE node
I20260926 20:40:49.690786 1087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:49.692417 1087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:49.693653 1087 hybrid_clock.cc:648] HybridClock initialized: now 1790455249693598 us; error 62 us; skew 500 ppm
I20260926 20:40:49.694051 1087 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:40:49.696451 1087 webserver.cc:460] Webserver started at http://127.23.44.254:39023/ using document root <none> and password file <none>
I20260926 20:40:49.697211 1087 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:49.697301 1087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:49.697611 1087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:49.702008 1087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "3318ce4670204bf4bfd0380957ca78eb"
format_stamp: "Formatted at 2026-09-26 20:40:49 on dist-test-slave-4d5t"
server_key: "627dee0c91ba455c92b326d01ed3f009"
server_key_iv: "f3d6ff612d5712709c1c8cfc49092d34"
server_key_version: "encryptionkey@0"
I20260926 20:40:49.702834 1087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "3318ce4670204bf4bfd0380957ca78eb"
format_stamp: "Formatted at 2026-09-26 20:40:49 on dist-test-slave-4d5t"
server_key: "627dee0c91ba455c92b326d01ed3f009"
server_key_iv: "f3d6ff612d5712709c1c8cfc49092d34"
server_key_version: "encryptionkey@0"
I20260926 20:40:49.709203 1087 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 20:40:49.714325 1101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:49.716027 1087 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:40:49.716286 1087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "3318ce4670204bf4bfd0380957ca78eb"
format_stamp: "Formatted at 2026-09-26 20:40:49 on dist-test-slave-4d5t"
server_key: "627dee0c91ba455c92b326d01ed3f009"
server_key_iv: "f3d6ff612d5712709c1c8cfc49092d34"
server_key_version: "encryptionkey@0"
I20260926 20:40:49.716634 1087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:49.723321 1087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:49.724138 1087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:49.724458 1087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:49.745252 1087 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:39593
I20260926 20:40:49.745298 1152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:39593 every 8 connection(s)
I20260926 20:40:49.746654 1087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:40:49.751101 1153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:49.752818 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 1087
I20260926 20:40:49.753044 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:40:49.753808 23731 external_mini_cluster.cc:1428] Setting key 4857c426bb906f76b8990cfa34f9da23
I20260926 20:40:49.762485 1156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:49.755963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54680 (local address 127.23.44.254:39593)
0926 20:40:49.757076 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:49.757096 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:49.757148 (+ 52us) server_negotiation.cc:407] Connection header received
0926 20:40:49.758902 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:49.758925 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:49.759317 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:49.759642 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:49.760836 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:49.760845 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:49.760857 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:49.760873 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:49.760941 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:49.760962 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:49.761170 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:49.761272 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 20:40:49.761593 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":419,"threads_started":1}
I20260926 20:40:49.763883 1153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb: Bootstrap starting.
I20260926 20:40:49.767076 1153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:49.768137 1153 log.cc:824] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:49.771145 1153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb: No bootstrap required, opened a new log
I20260926 20:40:49.779472 1153 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3318ce4670204bf4bfd0380957ca78eb" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39593 } }
I20260926 20:40:49.779850 1153 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:49.779971 1153 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3318ce4670204bf4bfd0380957ca78eb, State: Initialized, Role: FOLLOWER
I20260926 20:40:49.780695 1153 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3318ce4670204bf4bfd0380957ca78eb" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39593 } }
I20260926 20:40:49.780905 1153 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:40:49.781066 1153 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:40:49.781247 1153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:49.783699 1153 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3318ce4670204bf4bfd0380957ca78eb" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39593 } }
I20260926 20:40:49.784296 1153 leader_election.cc:302] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3318ce4670204bf4bfd0380957ca78eb; no voters:
I20260926 20:40:49.784915 1153 leader_election.cc:288] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:40:49.785080 1158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:49.785787 1158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [term 1 LEADER]: Becoming Leader. State: Replica: 3318ce4670204bf4bfd0380957ca78eb, State: Running, Role: LEADER
I20260926 20:40:49.786473 1158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3318ce4670204bf4bfd0380957ca78eb" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39593 } }
I20260926 20:40:49.787137 1153 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:40:49.789469 1160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3318ce4670204bf4bfd0380957ca78eb. Latest consensus state: current_term: 1 leader_uuid: "3318ce4670204bf4bfd0380957ca78eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3318ce4670204bf4bfd0380957ca78eb" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39593 } } }
I20260926 20:40:49.789935 1160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:49.791214 1159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3318ce4670204bf4bfd0380957ca78eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3318ce4670204bf4bfd0380957ca78eb" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 39593 } } }
I20260926 20:40:49.791651 1159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:49.792018 1167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:40:49.801136 1167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:40:49.809770 1167 catalog_manager.cc:1347] Generated new cluster ID: c16119b6fc0349c8ab4d79f3318cb59c
I20260926 20:40:49.809901 1167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:40:49.837867 1167 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:40:49.838327 1167 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:40:49.847337 1167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3318ce4670204bf4bfd0380957ca78eb: Generated new TSK 0
I20260926 20:40:49.848711 1167 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:40:49.860817 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39593
--builtin_ntp_servers=127.23.44.212:46227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 20:40:50.216463 1177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:50.216784 1177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:50.216950 1177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:50.224592 1177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:50.224704 1177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:50.224859 1177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:40:50.234355 1177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:46227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:50.235742 1177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:50.237617 1177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:50.251523 1182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:50.251482 1183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:50.251569 1185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:50.252739 1177 server_base.cc:1034] running on GCE node
I20260926 20:40:50.253448 1177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:50.254515 1177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:50.255748 1177 hybrid_clock.cc:648] HybridClock initialized: now 1790455250255688 us; error 55 us; skew 500 ppm
I20260926 20:40:50.256167 1177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:40:50.263055 1177 webserver.cc:460] Webserver started at http://127.23.44.193:44585/ using document root <none> and password file <none>
I20260926 20:40:50.263861 1177 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:50.264003 1177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:50.264331 1177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:50.267367 1177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "8d00e8c27e224d9a8c5eeafbc0ff6926"
format_stamp: "Formatted at 2026-09-26 20:40:50 on dist-test-slave-4d5t"
server_key: "68047430e9f9538648af03fb8fae32c2"
server_key_iv: "c687f115a87ac7f0f09a08c3d3b2a958"
server_key_version: "encryptionkey@0"
I20260926 20:40:50.268221 1177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "8d00e8c27e224d9a8c5eeafbc0ff6926"
format_stamp: "Formatted at 2026-09-26 20:40:50 on dist-test-slave-4d5t"
server_key: "68047430e9f9538648af03fb8fae32c2"
server_key_iv: "c687f115a87ac7f0f09a08c3d3b2a958"
server_key_version: "encryptionkey@0"
I20260926 20:40:50.277308 1177 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.002s sys 0.005s
I20260926 20:40:50.284261 1191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:50.286335 1177 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260926 20:40:50.286584 1177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "8d00e8c27e224d9a8c5eeafbc0ff6926"
format_stamp: "Formatted at 2026-09-26 20:40:50 on dist-test-slave-4d5t"
server_key: "68047430e9f9538648af03fb8fae32c2"
server_key_iv: "c687f115a87ac7f0f09a08c3d3b2a958"
server_key_version: "encryptionkey@0"
I20260926 20:40:50.286926 1177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:50.296610 1177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:50.297881 1177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:50.298255 1177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:50.299687 1177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:50.302680 1177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:50.302819 1177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:50.302938 1177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:50.303030 1177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:50.343238 1177 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:40325
I20260926 20:40:50.343289 1304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:40325 every 8 connection(s)
I20260926 20:40:50.345232 1177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:40:50.354696 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:50.348543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:58393 (local address 127.23.44.254:39593)
0926 20:40:50.349054 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:50.349058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:50.350648 (+ 1590us) server_negotiation.cc:407] Connection header received
0926 20:40:50.352368 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:50.352375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:50.352441 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:50.352562 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:50.353915 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:50.353920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:50.353925 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:50.353943 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:50.353997 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:50.354008 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:50.354143 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:50.354327 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 20:40:50.354395 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":256,"threads_started":1}
I20260926 20:40:50.355643 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 1177
I20260926 20:40:50.355865 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:40:50.355850 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:50.349307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39593 (local address 127.23.44.193:58393)
0926 20:40:50.350490 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:50.350528 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:50.352066 (+ 1538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:50.352814 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:50.352821 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:50.352910 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:50.353623 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:40:50.353626 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:50.353668 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:40:50.353751 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:50.354103 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:50.354113 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:50.354121 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 20:40:50.354402 (+ 281us) client_negotiation.cc:240] Negotiation successful
0926 20:40:50.354658 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":347,"threads_started":1}
I20260926 20:40:50.356698 23731 external_mini_cluster.cc:1428] Setting key 422e5e1ac3d379ac628529d1a58418e8
I20260926 20:40:50.357602 1305 heartbeater.cc:347] Connected to a master server at 127.23.44.254:39593
I20260926 20:40:50.357919 1305 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:50.358618 1305 heartbeater.cc:511] Master 127.23.44.254:39593 requested a full tablet report, sending...
I20260926 20:40:50.359815 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39593
--builtin_ntp_servers=127.23.44.212:46227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 20:40:50.361409 1118 ts_manager.cc:194] Registered new tserver with Master: 8d00e8c27e224d9a8c5eeafbc0ff6926 (127.23.44.193:40325)
W20260926 20:40:50.712126 1309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:50.712491 1309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:50.712670 1309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:50.721469 1309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:50.721638 1309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:50.721776 1309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:40:50.731562 1309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:46227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:50.732920 1309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:50.734707 1309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:50.748811 1315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:50.749130 1314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:50.749368 1317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:50.750161 1309 server_base.cc:1034] running on GCE node
I20260926 20:40:50.750891 1309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:50.752318 1309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:50.753558 1309 hybrid_clock.cc:648] HybridClock initialized: now 1790455250753506 us; error 42 us; skew 500 ppm
I20260926 20:40:50.753961 1309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:40:50.756929 1309 webserver.cc:460] Webserver started at http://127.23.44.194:43831/ using document root <none> and password file <none>
I20260926 20:40:50.757844 1309 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:50.757961 1309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:50.758271 1309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:50.762253 1309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "c3862fb0fcaa485887b77b97af4e78d1"
format_stamp: "Formatted at 2026-09-26 20:40:50 on dist-test-slave-4d5t"
server_key: "b71093dd85bcb37b393f952b450c177d"
server_key_iv: "e520c362fd8e52ba8b2d29d1685edd9d"
server_key_version: "encryptionkey@0"
I20260926 20:40:50.763289 1309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "c3862fb0fcaa485887b77b97af4e78d1"
format_stamp: "Formatted at 2026-09-26 20:40:50 on dist-test-slave-4d5t"
server_key: "b71093dd85bcb37b393f952b450c177d"
server_key_iv: "e520c362fd8e52ba8b2d29d1685edd9d"
server_key_version: "encryptionkey@0"
I20260926 20:40:50.769416 1309 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260926 20:40:50.772938 1323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:50.774477 1309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:40:50.774695 1309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "c3862fb0fcaa485887b77b97af4e78d1"
format_stamp: "Formatted at 2026-09-26 20:40:50 on dist-test-slave-4d5t"
server_key: "b71093dd85bcb37b393f952b450c177d"
server_key_iv: "e520c362fd8e52ba8b2d29d1685edd9d"
server_key_version: "encryptionkey@0"
I20260926 20:40:50.774966 1309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:50.781561 1309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:50.783004 1309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:50.783339 1309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:50.784421 1309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:50.786170 1309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:50.786309 1309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:50.786456 1309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:50.786554 1309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:50.824788 1309 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:45949
I20260926 20:40:50.824735 1435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:45949 every 8 connection(s)
I20260926 20:40:50.826118 1309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:40:50.834403 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:50.828889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:40687 (local address 127.23.44.254:39593)
0926 20:40:50.829069 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:50.829073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:50.830457 (+ 1384us) server_negotiation.cc:407] Connection header received
0926 20:40:50.832106 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:50.832110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:50.832164 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:50.832257 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:50.833686 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:50.833690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:50.833694 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:50.833709 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:50.833763 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:50.833775 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:50.833899 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:50.834120 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 20:40:50.834171 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 20:40:50.834800 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 1309
I20260926 20:40:50.835028 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:40:50.835656 23731 external_mini_cluster.cc:1428] Setting key 9d3ab9f7af9699511315bf016f263d57
I20260926 20:40:50.835913 1438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:50.829350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39593 (local address 127.23.44.194:40687)
0926 20:40:50.830288 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:50.830323 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:50.831846 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:50.832479 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:50.832487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:50.832659 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:50.833393 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:40:50.833396 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:50.833435 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:40:50.833521 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:50.833913 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:50.833923 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:50.833930 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 20:40:50.834191 (+ 261us) client_negotiation.cc:240] Negotiation successful
0926 20:40:50.834506 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":300,"threads_started":1}
I20260926 20:40:50.838205 1436 heartbeater.cc:347] Connected to a master server at 127.23.44.254:39593
I20260926 20:40:50.838536 1436 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:50.839125 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39593
--builtin_ntp_servers=127.23.44.212:46227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 20:40:50.839144 1436 heartbeater.cc:511] Master 127.23.44.254:39593 requested a full tablet report, sending...
I20260926 20:40:50.840744 1118 ts_manager.cc:194] Registered new tserver with Master: c3862fb0fcaa485887b77b97af4e78d1 (127.23.44.194:45949)
W20260926 20:40:51.191401 1439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:51.191723 1439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:51.191824 1439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:51.200459 1439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:51.200610 1439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:51.200733 1439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:40:51.211261 1439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:46227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:39593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:51.212679 1439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:51.214514 1439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:51.229210 1444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:51.229127 1445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:51.229156 1447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:51.230223 1439 server_base.cc:1034] running on GCE node
I20260926 20:40:51.230865 1439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:51.231879 1439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:51.233194 1439 hybrid_clock.cc:648] HybridClock initialized: now 1790455251233108 us; error 36 us; skew 500 ppm
I20260926 20:40:51.233855 1439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:40:51.236492 1439 webserver.cc:460] Webserver started at http://127.23.44.195:43771/ using document root <none> and password file <none>
I20260926 20:40:51.237263 1439 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:51.237406 1439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:51.237737 1439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:51.240789 1439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "1cca58466d264224bf32360735e52e9e"
format_stamp: "Formatted at 2026-09-26 20:40:51 on dist-test-slave-4d5t"
server_key: "e143d62647bcba68467c8e2db096dda1"
server_key_iv: "35e72f6f511ddb947b69c88853dbf435"
server_key_version: "encryptionkey@0"
I20260926 20:40:51.241668 1439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "1cca58466d264224bf32360735e52e9e"
format_stamp: "Formatted at 2026-09-26 20:40:51 on dist-test-slave-4d5t"
server_key: "e143d62647bcba68467c8e2db096dda1"
server_key_iv: "35e72f6f511ddb947b69c88853dbf435"
server_key_version: "encryptionkey@0"
I20260926 20:40:51.247148 1439 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260926 20:40:51.251217 1453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:51.252775 1439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 20:40:51.252965 1439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "1cca58466d264224bf32360735e52e9e"
format_stamp: "Formatted at 2026-09-26 20:40:51 on dist-test-slave-4d5t"
server_key: "e143d62647bcba68467c8e2db096dda1"
server_key_iv: "35e72f6f511ddb947b69c88853dbf435"
server_key_version: "encryptionkey@0"
I20260926 20:40:51.253196 1439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:51.259109 1439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:51.259956 1439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:51.260233 1439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:51.261314 1439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:51.262753 1439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:51.262885 1439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:51.263011 1439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:51.263098 1439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:51.300544 1439 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:34445
I20260926 20:40:51.300694 1565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:34445 every 8 connection(s)
I20260926 20:40:51.302845 1439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:40:51.304720 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 1439
I20260926 20:40:51.304911 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:40:51.305377 23731 external_mini_cluster.cc:1428] Setting key cb69fc0c6d9690426c56a4079abcf78b
I20260926 20:40:51.311914 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:51.305761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:50945 (local address 127.23.44.254:39593)
0926 20:40:51.305975 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:51.305981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:51.308415 (+ 2434us) server_negotiation.cc:407] Connection header received
0926 20:40:51.309613 (+ 1198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:51.309619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:51.309686 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:51.309807 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:51.311161 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:51.311167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:51.311171 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:51.311190 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:51.311231 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:51.311243 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:51.311363 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:51.311565 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 20:40:51.311632 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 20:40:51.313020 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:51.306409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:39593 (local address 127.23.44.195:50945)
0926 20:40:51.307514 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:51.307551 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:51.309351 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:51.310043 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:51.310052 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:51.310146 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:51.310857 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:40:51.310860 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:51.310907 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:40:51.310993 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:51.311347 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:51.311357 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:51.311364 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 20:40:51.311652 (+ 288us) client_negotiation.cc:240] Negotiation successful
0926 20:40:51.311927 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":420,"threads_started":1}
I20260926 20:40:51.314385 1566 heartbeater.cc:347] Connected to a master server at 127.23.44.254:39593
I20260926 20:40:51.314718 1566 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:51.315449 1566 heartbeater.cc:511] Master 127.23.44.254:39593 requested a full tablet report, sending...
I20260926 20:40:51.317054 1118 ts_manager.cc:194] Registered new tserver with Master: 1cca58466d264224bf32360735e52e9e (127.23.44.195:34445)
I20260926 20:40:51.321663 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 20:40:51.328557 1307 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:40:51.326689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54696 (local address 127.23.44.254:39593)
0926 20:40:51.326893 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:51.326899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:51.327518 (+ 619us) server_negotiation.cc:407] Connection header received
0926 20:40:51.327709 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:51.327714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:51.327771 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:51.327869 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:51.328180 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54696: BlockingRecv error: recv got EOF from 127.0.0.1:54696 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:40:51.329028 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 1177
I20260926 20:40:51.333180 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:51.330286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53314 (local address 127.23.44.193:40325)
0926 20:40:51.330867 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:51.330874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:51.330901 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:40:51.330960 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:51.330966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:51.331195 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:51.331327 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:51.332227 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:51.332235 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:51.332246 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:51.332271 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:51.332316 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:51.332331 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:51.332467 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:51.332841 (+ 374us) server_negotiation.cc:299] Negotiation successful
0926 20:40:51.332923 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":298,"threads_started":1}
I20260926 20:40:51.334503 1300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:51.414891 1305 heartbeater.cc:503] Master 127.23.44.254:39593 was elected leader, sending a full tablet report...
I20260926 20:40:51.416241 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 1177
W20260926 20:40:51.432957 1102 connection.cc:535] server connection from 127.23.44.193:58393 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 20:40:51.433251 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 1309
I20260926 20:40:51.437561 1577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:51.434182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.23.44.194:45949)
0926 20:40:51.434671 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:51.434684 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:51.434711 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:40:51.434782 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:51.434789 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:51.435027 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:51.435371 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:51.436789 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:51.436796 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:51.436807 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:51.436825 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:51.436864 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:51.436878 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:51.437041 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:51.437187 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 20:40:51.437287 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":256,"threads_started":1}
I20260926 20:40:51.439034 1431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:51.522989 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 1309
I20260926 20:40:51.540604 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 1439
I20260926 20:40:51.544667 1579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:51.541686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49548 (local address 127.23.44.195:34445)
0926 20:40:51.542377 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:51.542384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:51.542415 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:40:51.542476 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:51.542482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:51.542705 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:51.542844 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:51.543879 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:51.543886 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:51.543898 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:40:51.543914 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:51.543952 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:40:51.543965 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:51.544096 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:51.544156 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 20:40:51.544266 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":385,"threads_started":1}
I20260926 20:40:51.545974 1561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:51.631099 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 1439
I20260926 20:40:51.647732 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 1087
I20260926 20:40:51.649434 1148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:51.753966 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 1087
2026-09-26T20:40:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2515 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:40:51 dist-test-slave-4d5t krb5kdc[1584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:40:51 dist-test-slave-4d5t krb5kdc[1584](info): set up 2 sockets
Sep 26 20:40:51 dist-test-slave-4d5t krb5kdc[1584](info): commencing operation
krb5kdc: starting...
W20260926 20:40:52.885818 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:40:53 dist-test-slave-4d5t krb5kdc[1584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:40:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:40:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:40:53.141230 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:42809
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:42809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:53.521145 1597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:53.521445 1597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:53.521647 1597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:53.529284 1597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:40:53.529466 1597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:53.529563 1597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:53.529623 1597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:40:53.529690 1597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:40:53.540959 1597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:42809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:42809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:53.542582 1597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:53.543934 1597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:53.557590 1602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:53.558036 1605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:53.558075 1603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:53.559177 1597 server_base.cc:1034] running on GCE node
I20260926 20:40:53.560056 1597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:53.561801 1597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:53.563077 1597 hybrid_clock.cc:648] HybridClock initialized: now 1790455253563027 us; error 68 us; skew 500 ppm
I20260926 20:40:53.563485 1597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:53 dist-test-slave-4d5t krb5kdc[1584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455253, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:53.567201 1597 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:40:53.569301 1597 webserver.cc:460] Webserver started at http://127.23.44.254:38481/ using document root <none> and password file <none>
I20260926 20:40:53.570044 1597 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:53.570199 1597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:53.570524 1597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:53.573690 1597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "55e8d7484d314ed5978d95c55bf67074"
format_stamp: "Formatted at 2026-09-26 20:40:53 on dist-test-slave-4d5t"
server_key: "2da657acd2808952c3c62c9becbaa137"
server_key_iv: "97baea2e07e2cc243b69fd4f815cbf52"
server_key_version: "encryptionkey@0"
I20260926 20:40:53.574469 1597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "55e8d7484d314ed5978d95c55bf67074"
format_stamp: "Formatted at 2026-09-26 20:40:53 on dist-test-slave-4d5t"
server_key: "2da657acd2808952c3c62c9becbaa137"
server_key_iv: "97baea2e07e2cc243b69fd4f815cbf52"
server_key_version: "encryptionkey@0"
I20260926 20:40:53.580243 1597 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:40:53.585371 1612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:53.587211 1597 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260926 20:40:53.587484 1597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "55e8d7484d314ed5978d95c55bf67074"
format_stamp: "Formatted at 2026-09-26 20:40:53 on dist-test-slave-4d5t"
server_key: "2da657acd2808952c3c62c9becbaa137"
server_key_iv: "97baea2e07e2cc243b69fd4f815cbf52"
server_key_version: "encryptionkey@0"
I20260926 20:40:53.587735 1597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:53.618934 1597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:53.623975 1597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:53.624320 1597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:53.647006 1597 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:42809
I20260926 20:40:53.647043 1663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:42809 every 8 connection(s)
I20260926 20:40:53.648861 1597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:40:53.653138 1664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:53.656903 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 1597
I20260926 20:40:53.657160 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:40:53.657763 23731 external_mini_cluster.cc:1428] Setting key 078c7d86f8aaa378e9ec06b1c6908b1d
I20260926 20:40:53.664705 1664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074: Bootstrap starting.
I20260926 20:40:53.668391 1664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:53.669592 1664 log.cc:824] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:53.672484 1664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074: No bootstrap required, opened a new log
Sep 26 20:40:53 dist-test-slave-4d5t krb5kdc[1584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:40:53.681651 1664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e8d7484d314ed5978d95c55bf67074" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 42809 } }
I20260926 20:40:53.682039 1664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:53.682170 1664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e8d7484d314ed5978d95c55bf67074, State: Initialized, Role: FOLLOWER
I20260926 20:40:53.682699 1667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:53.660077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37680 (local address 127.23.44.254:42809)
0926 20:40:53.660999 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:53.661022 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:53.661066 (+ 44us) server_negotiation.cc:407] Connection header received
0926 20:40:53.662978 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:53.663005 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:53.663438 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:53.663785 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:53.665699 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:53.667083 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:53.668965 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:53.669714 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:53.669936 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:53.674208 (+ 4272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:53.674216 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:53.674231 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:53.674272 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:53.677610 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:53.678438 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:53.678444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:53.678451 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:53.678619 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:53.679005 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:53.679008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:53.679010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:53.679557 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:53.679795 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:53.680229 (+ 434us) server_negotiation.cc:299] Negotiation successful
0926 20:40:53.680950 (+ 721us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":466,"threads_started":1}
I20260926 20:40:53.682821 1664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e8d7484d314ed5978d95c55bf67074" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 42809 } }
I20260926 20:40:53.683168 1664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:40:53.683349 1664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:40:53.683544 1664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:53.685966 1664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e8d7484d314ed5978d95c55bf67074" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 42809 } }
I20260926 20:40:53.686523 1664 leader_election.cc:302] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55e8d7484d314ed5978d95c55bf67074; no voters:
I20260926 20:40:53.687155 1664 leader_election.cc:288] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:40:53.687352 1669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:53.688505 1669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [term 1 LEADER]: Becoming Leader. State: Replica: 55e8d7484d314ed5978d95c55bf67074, State: Running, Role: LEADER
I20260926 20:40:53.689282 1664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:40:53.689234 1669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e8d7484d314ed5978d95c55bf67074" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 42809 } }
I20260926 20:40:53.692719 1670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55e8d7484d314ed5978d95c55bf67074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e8d7484d314ed5978d95c55bf67074" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 42809 } } }
I20260926 20:40:53.693176 1670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:53.693786 1671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55e8d7484d314ed5978d95c55bf67074. Latest consensus state: current_term: 1 leader_uuid: "55e8d7484d314ed5978d95c55bf67074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55e8d7484d314ed5978d95c55bf67074" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 42809 } } }
I20260926 20:40:53.694186 1671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074 [sys.catalog]: This master's current role is: LEADER
I20260926 20:40:53.694156 1678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:40:53.702962 1678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:40:53.716357 1678 catalog_manager.cc:1347] Generated new cluster ID: 56677994f21b40f4b45bff9c5c053005
I20260926 20:40:53.716500 1678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:40:53.728545 1678 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:40:53.730242 1678 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:40:53.737588 1678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074: Generated new TSK 0
I20260926 20:40:53.738484 1678 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:40:53.808092 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:42809
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:54.190635 1690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:54.191138 1690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:54.191306 1690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:54.199049 1690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:54.199162 1690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:54.199277 1690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:40:54.208999 1690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:42809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:54.210234 1690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:54.211469 1690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:54.224903 1696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:54.225063 1695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:54.225078 1698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:54.226104 1690 server_base.cc:1034] running on GCE node
I20260926 20:40:54.226890 1690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:54.227900 1690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:54.229127 1690 hybrid_clock.cc:648] HybridClock initialized: now 1790455254229071 us; error 51 us; skew 500 ppm
I20260926 20:40:54.229449 1690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:54 dist-test-slave-4d5t krb5kdc[1584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455254, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:54.232811 1690 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:40:54.234706 1690 webserver.cc:460] Webserver started at http://127.23.44.193:41355/ using document root <none> and password file <none>
I20260926 20:40:54.235452 1690 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:54.235596 1690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:54.235948 1690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:54.239179 1690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "55b1de103c914f4c849343715aa0d0bc"
format_stamp: "Formatted at 2026-09-26 20:40:54 on dist-test-slave-4d5t"
server_key: "2fa2036ed3597b68b2743d743bd10a32"
server_key_iv: "2443d12ee1447535569ef75cefd5b27a"
server_key_version: "encryptionkey@0"
I20260926 20:40:54.239936 1690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "55b1de103c914f4c849343715aa0d0bc"
format_stamp: "Formatted at 2026-09-26 20:40:54 on dist-test-slave-4d5t"
server_key: "2fa2036ed3597b68b2743d743bd10a32"
server_key_iv: "2443d12ee1447535569ef75cefd5b27a"
server_key_version: "encryptionkey@0"
I20260926 20:40:54.245572 1690 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 20:40:54.249279 1705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:54.250715 1690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:40:54.250902 1690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "55b1de103c914f4c849343715aa0d0bc"
format_stamp: "Formatted at 2026-09-26 20:40:54 on dist-test-slave-4d5t"
server_key: "2fa2036ed3597b68b2743d743bd10a32"
server_key_iv: "2443d12ee1447535569ef75cefd5b27a"
server_key_version: "encryptionkey@0"
I20260926 20:40:54.251118 1690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:54.272334 1690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:54.276867 1690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:54.277184 1690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:54.278205 1690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:54.279687 1690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:54.279800 1690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:54.279959 1690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:54.280035 1690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:54.316210 1690 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:36495
I20260926 20:40:54.316264 1817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:36495 every 8 connection(s)
I20260926 20:40:54.317555 1690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:40:54.318719 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 1690
I20260926 20:40:54.318969 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:40:54.319741 23731 external_mini_cluster.cc:1428] Setting key 05882944f9735142985e175e11fb2018
Sep 26 20:40:54 dist-test-slave-4d5t krb5kdc[1584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455254, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:40:54.340616 1820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:54.320577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:35325 (local address 127.23.44.254:42809)
0926 20:40:54.321111 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:54.321117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:54.322250 (+ 1133us) server_negotiation.cc:407] Connection header received
0926 20:40:54.324104 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:54.324112 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:54.324202 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:54.324318 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:54.325935 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:54.327092 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:54.328760 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:54.329176 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:54.329272 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:54.333771 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:54.333777 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:54.333780 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:54.333812 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:54.337000 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:54.337803 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:54.337811 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:54.337819 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:54.337946 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:54.338400 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:54.338404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:54.338412 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:54.338717 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:54.338888 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:54.339894 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0926 20:40:54.340156 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":263,"threads_started":1}
I20260926 20:40:54.341688 1821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:54.321231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:42809 (local address 127.23.44.193:35325)
0926 20:40:54.322084 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:54.322129 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:54.323822 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:54.324540 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:54.324600 (+ 60us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:54.325148 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:54.325745 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:54.325762 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:54.327261 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:54.327265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:54.328544 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:54.328621 (+ 77us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:54.329418 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:54.329427 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:54.329786 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:54.330649 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:54.330689 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:54.333501 (+ 2812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:54.337163 (+ 3662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:54.337173 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:54.337194 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:54.337661 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:54.338062 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:54.338067 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:54.338071 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:54.338274 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:54.338849 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:54.338859 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:54.339332 (+ 473us) client_negotiation.cc:769] Sending connection context
0926 20:40:54.339780 (+ 448us) client_negotiation.cc:240] Negotiation successful
0926 20:40:54.340414 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":367,"threads_started":1}
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
I20260926 20:40:54.343357 1818 heartbeater.cc:347] Connected to a master server at 127.23.44.254:42809
I20260926 20:40:54.343698 1818 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:54.344624 1818 heartbeater.cc:511] Master 127.23.44.254:42809 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
I20260926 20:40:54.346915 1629 ts_manager.cc:194] Registered new tserver with Master: 55b1de103c914f4c849343715aa0d0bc (127.23.44.193:36495)
I20260926 20:40:54.348412 1629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:35325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:40:54.375631 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:42809
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:54.711514 1824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:54.711812 1824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:54.711978 1824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:54.719269 1824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:54.719403 1824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:54.719597 1824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:40:54.729496 1824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:42809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:54.730844 1824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:54.732170 1824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:54.746368 1829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:54.746433 1830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:54.746397 1832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:54.747272 1824 server_base.cc:1034] running on GCE node
I20260926 20:40:54.747905 1824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:54.748935 1824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:54.750177 1824 hybrid_clock.cc:648] HybridClock initialized: now 1790455254750104 us; error 71 us; skew 500 ppm
I20260926 20:40:54.750521 1824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:54 dist-test-slave-4d5t krb5kdc[1584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455254, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:54.754058 1824 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:40:54.756124 1824 webserver.cc:460] Webserver started at http://127.23.44.194:46567/ using document root <none> and password file <none>
I20260926 20:40:54.757011 1824 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:54.757164 1824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:54.757480 1824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:54.760516 1824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "8dacd89ad02b436ab74fb7314834e913"
format_stamp: "Formatted at 2026-09-26 20:40:54 on dist-test-slave-4d5t"
server_key: "b68413705be7c2f2d34312f6cb8f4979"
server_key_iv: "994d14a11e9dbcdc79f1ea532a289f25"
server_key_version: "encryptionkey@0"
I20260926 20:40:54.761415 1824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "8dacd89ad02b436ab74fb7314834e913"
format_stamp: "Formatted at 2026-09-26 20:40:54 on dist-test-slave-4d5t"
server_key: "b68413705be7c2f2d34312f6cb8f4979"
server_key_iv: "994d14a11e9dbcdc79f1ea532a289f25"
server_key_version: "encryptionkey@0"
I20260926 20:40:54.767117 1824 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:40:54.771081 1839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:54.772799 1824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:40:54.773020 1824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "8dacd89ad02b436ab74fb7314834e913"
format_stamp: "Formatted at 2026-09-26 20:40:54 on dist-test-slave-4d5t"
server_key: "b68413705be7c2f2d34312f6cb8f4979"
server_key_iv: "994d14a11e9dbcdc79f1ea532a289f25"
server_key_version: "encryptionkey@0"
I20260926 20:40:54.773265 1824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:54.803083 1824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:54.807730 1824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:54.808065 1824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:54.809216 1824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:54.810695 1824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:54.810799 1824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:54.810948 1824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:54.811025 1824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:54.849444 1824 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:36723
I20260926 20:40:54.849505 1951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:36723 every 8 connection(s)
I20260926 20:40:54.850867 1824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:40:54.852671 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 1824
I20260926 20:40:54.852908 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:40:54.853461 23731 external_mini_cluster.cc:1428] Setting key 9cae395a71cde8d8f96938dce1a56353
Sep 26 20:40:54 dist-test-slave-4d5t krb5kdc[1584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455254, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:40:54.874557 1954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:54.854023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:52477 (local address 127.23.44.254:42809)
0926 20:40:54.854529 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:54.854536 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:54.855728 (+ 1192us) server_negotiation.cc:407] Connection header received
0926 20:40:54.857852 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:54.857858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:54.857963 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:54.858104 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:54.859780 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:54.860680 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:54.862475 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:54.862943 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:54.863091 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:54.867637 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:54.867642 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:54.867646 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:54.867680 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:54.870814 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:54.871772 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:54.871780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:54.871784 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:54.871941 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:54.872450 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:54.872456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:54.872462 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:54.872827 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:54.873005 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:54.873910 (+ 905us) server_negotiation.cc:299] Negotiation successful
0926 20:40:54.874157 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":249,"threads_started":1}
I20260926 20:40:54.875665 1955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:54.854630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:42809 (local address 127.23.44.194:52477)
0926 20:40:54.855513 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:54.855565 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:54.857566 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:54.858360 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:54.858375 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:54.858956 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:54.859585 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:54.859604 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:54.860875 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:54.860886 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:54.862289 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:54.862306 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:54.863192 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:54.863199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:54.863568 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:54.864395 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:54.864433 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:54.867367 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:54.871014 (+ 3647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:54.871025 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:54.871045 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:54.871522 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:54.872078 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:54.872084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:54.872089 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:54.872302 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:54.872980 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:54.872990 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:54.873486 (+ 496us) client_negotiation.cc:769] Sending connection context
0926 20:40:54.873789 (+ 303us) client_negotiation.cc:240] Negotiation successful
0926 20:40:54.874400 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":330,"threads_started":1}
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
I20260926 20:40:54.877362 1952 heartbeater.cc:347] Connected to a master server at 127.23.44.254:42809
I20260926 20:40:54.877677 1952 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:54.878556 1952 heartbeater.cc:511] Master 127.23.44.254:42809 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
I20260926 20:40:54.880147 1629 ts_manager.cc:194] Registered new tserver with Master: 8dacd89ad02b436ab74fb7314834e913 (127.23.44.194:36723)
I20260926 20:40:54.881384 1629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:52477
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:40:54.912910 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:42809
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:40:55.257942 1958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:40:55.258283 1958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:40:55.258451 1958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:40:55.265995 1958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:40:55.266155 1958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:40:55.266340 1958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:40:55.276415 1958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:42809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:40:55.277863 1958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:40:55.279176 1958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:40:55.293171 1963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:55.293151 1964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:40:55.293406 1966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:40:55.294351 1958 server_base.cc:1034] running on GCE node
I20260926 20:40:55.295042 1958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:40:55.296243 1958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:40:55.297448 1958 hybrid_clock.cc:648] HybridClock initialized: now 1790455255297369 us; error 68 us; skew 500 ppm
I20260926 20:40:55.297839 1958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:40:55 dist-test-slave-4d5t krb5kdc[1584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:40:55.308286 1958 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:40:55.310173 1958 webserver.cc:460] Webserver started at http://127.23.44.195:42037/ using document root <none> and password file <none>
I20260926 20:40:55.311056 1958 fs_manager.cc:359] Metadata directory not provided
I20260926 20:40:55.311244 1958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:40:55.311587 1958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:40:55.314858 1958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "501b9a732c99493b9bc0cab8c861b434"
format_stamp: "Formatted at 2026-09-26 20:40:55 on dist-test-slave-4d5t"
server_key: "fdca058bbf5db8c74a5be14398f09fed"
server_key_iv: "e8960d4c4f0d70802f31ba3761f51118"
server_key_version: "encryptionkey@0"
I20260926 20:40:55.315894 1958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "501b9a732c99493b9bc0cab8c861b434"
format_stamp: "Formatted at 2026-09-26 20:40:55 on dist-test-slave-4d5t"
server_key: "fdca058bbf5db8c74a5be14398f09fed"
server_key_iv: "e8960d4c4f0d70802f31ba3761f51118"
server_key_version: "encryptionkey@0"
I20260926 20:40:55.322798 1958 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260926 20:40:55.327792 1973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:55.329876 1958 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 20:40:55.330237 1958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "501b9a732c99493b9bc0cab8c861b434"
format_stamp: "Formatted at 2026-09-26 20:40:55 on dist-test-slave-4d5t"
server_key: "fdca058bbf5db8c74a5be14398f09fed"
server_key_iv: "e8960d4c4f0d70802f31ba3761f51118"
server_key_version: "encryptionkey@0"
I20260926 20:40:55.330562 1958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:40:55.352524 1818 heartbeater.cc:503] Master 127.23.44.254:42809 was elected leader, sending a full tablet report...
I20260926 20:40:55.353619 1958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:40:55.359153 1958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:40:55.359486 1958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:40:55.360630 1958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:40:55.362134 1958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:40:55.362213 1958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:55.362399 1958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:40:55.362457 1958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:40:55.400349 1958 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:45363
I20260926 20:40:55.400437 2085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:45363 every 8 connection(s)
I20260926 20:40:55.401736 1958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:40:55.408766 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 1958
I20260926 20:40:55.408994 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:40:55.409435 23731 external_mini_cluster.cc:1428] Setting key d7e02fa1957792ed6071cb69b2dab5c7
Sep 26 20:40:55 dist-test-slave-4d5t krb5kdc[1584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455255, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:40:55.425778 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.404687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:54077 (local address 127.23.44.254:42809)
0926 20:40:55.405238 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:55.405244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:55.406159 (+ 915us) server_negotiation.cc:407] Connection header received
0926 20:40:55.408144 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:55.408150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:55.408252 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:55.408396 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:55.411345 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.412649 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.414785 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.415266 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.415391 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.419251 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:55.419257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:55.419261 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:55.419296 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:55.422215 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:55.423094 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:55.423099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:55.423101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:55.423223 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:55.423678 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:55.423681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:55.423683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:55.423948 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:55.424123 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:55.425133 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0926 20:40:55.425362 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":284,"threads_started":1}
I20260926 20:40:55.427022 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.405254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:42809 (local address 127.23.44.195:54077)
0926 20:40:55.405985 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:55.406030 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:55.407847 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:55.408742 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:55.408752 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:55.409857 (+ 1105us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:40:55.410977 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:55.411017 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.412854 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.412862 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:55.414586 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:55.414606 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.415517 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.415524 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:55.415750 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.416438 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:40:55.416467 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:40:55.418898 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:40:55.422418 (+ 3520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:55.422426 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:55.422443 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:55.422946 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:55.423340 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:40:55.423343 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:40:55.423346 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:40:55.423566 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:40:55.424084 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:40:55.424091 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:40:55.424653 (+ 562us) client_negotiation.cc:769] Sending connection context
0926 20:40:55.425006 (+ 353us) client_negotiation.cc:240] Negotiation successful
0926 20:40:55.425639 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":284,"threads_started":1}
I20260926 20:40:55.428511 2086 heartbeater.cc:347] Connected to a master server at 127.23.44.254:42809
I20260926 20:40:55.429177 2086 heartbeater.cc:464] Registering TS with master...
I20260926 20:40:55.430251 2086 heartbeater.cc:511] Master 127.23.44.254:42809 requested a full tablet report, sending...
I20260926 20:40:55.432247 1629 ts_manager.cc:194] Registered new tserver with Master: 501b9a732c99493b9bc0cab8c861b434 (127.23.44.195:45363)
I20260926 20:40:55.433267 1629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:54077
I20260926 20:40:55.439544 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:40:55.458737 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.444023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.23.44.254:42809)
0926 20:40:55.444223 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:55.444232 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:55.444700 (+ 468us) server_negotiation.cc:407] Connection header received
0926 20:40:55.444908 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:55.444919 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:55.444983 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:55.445090 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:55.446471 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.447193 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.449138 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.449592 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.449754 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.452546 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:55.452597 (+ 51us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:55.452605 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:55.452662 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:55.455594 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:55.456389 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:55.456398 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:55.456402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:55.456632 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:55.457050 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:55.457057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:55.457061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:55.457444 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:55.457719 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:55.458115 (+ 396us) server_negotiation.cc:299] Negotiation successful
0926 20:40:55.458368 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 20:40:55.463088 1629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:40:55.469851 1629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:40:55.505214 2099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.484253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:36495 (local address 127.0.0.1:35604)
0926 20:40:55.497903 (+ 13650us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:55.497923 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:55.498039 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:55.498790 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:55.498798 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:55.498829 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:55.499310 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:55.499321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.500927 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.500934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:55.502653 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:55.502666 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.504110 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.504117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:55.504327 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.504387 (+ 60us) client_negotiation.cc:769] Sending connection context
0926 20:40:55.504554 (+ 167us) client_negotiation.cc:240] Negotiation successful
0926 20:40:55.504799 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13518,"thread_start_us":11585,"threads_started":1}
I20260926 20:40:55.505192 2101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.484469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.23.44.193:36495)
0926 20:40:55.497131 (+ 12662us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:55.497147 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:55.498219 (+ 1072us) server_negotiation.cc:407] Connection header received
0926 20:40:55.498328 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:55.498335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:55.498558 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:55.498788 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:55.499527 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.500777 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.502826 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.503874 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.504048 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.504317 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:55.504629 (+ 312us) server_negotiation.cc:299] Negotiation successful
0926 20:40:55.504815 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12405,"thread_start_us":12696,"threads_started":1}
I20260926 20:40:55.509625 1753 tablet_service.cc:1467] Processing CreateTablet for tablet 26b72525b8424a9fa6cb1a93de3ba9eb (DEFAULT_TABLE table=test-table [id=1db3c7f6caaf44a9bb4f4c5700cf7436]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:55.511540 1753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b72525b8424a9fa6cb1a93de3ba9eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:55.517570 2104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.488804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52240 (local address 127.23.44.194:36723)
0926 20:40:55.509079 (+ 20275us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:55.509089 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:55.509119 (+ 30us) server_negotiation.cc:407] Connection header received
0926 20:40:55.509220 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:55.509227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:55.509497 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:55.509716 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:55.510779 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.512460 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.514405 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.516047 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.516212 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.516434 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:55.517064 (+ 630us) server_negotiation.cc:299] Negotiation successful
0926 20:40:55.517245 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20103,"thread_start_us":19951,"threads_started":1}
I20260926 20:40:55.517571 2099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.488177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:36723 (local address 127.0.0.1:52240)
0926 20:40:55.505967 (+ 17790us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:55.505988 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:55.506089 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:55.510350 (+ 4261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:55.510355 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:55.510385 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:55.510583 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:55.510595 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.512667 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.512676 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:55.514196 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:55.514214 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.516581 (+ 2367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.516589 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:55.516834 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.516880 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 20:40:55.516998 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 20:40:55.517169 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17665,"thread_start_us":21903,"threads_started":1}
I20260926 20:40:55.520274 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.484127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:45363 (local address 127.0.0.1:42738)
0926 20:40:55.497387 (+ 13260us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:55.497463 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:55.497993 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:55.513822 (+ 15829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:55.513827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:55.513855 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:55.514286 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:55.514299 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.516100 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.516108 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:55.517972 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:55.517989 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.519368 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.519376 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:55.519587 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.519635 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 20:40:55.519742 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 20:40:55.519899 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11951,"thread_start_us":13466,"threads_started":1}
I20260926 20:40:55.521127 2102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.484438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42738 (local address 127.23.44.195:45363)
0926 20:40:55.513151 (+ 28713us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:55.513161 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:55.513198 (+ 37us) server_negotiation.cc:407] Connection header received
0926 20:40:55.513306 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:55.513314 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:55.513574 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:55.513803 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:55.514476 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.515969 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.518187 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.519141 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.519279 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.519521 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:55.520590 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0926 20:40:55.520767 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28527,"thread_start_us":28366,"threads_started":1}
I20260926 20:40:55.522152 1887 tablet_service.cc:1467] Processing CreateTablet for tablet 26b72525b8424a9fa6cb1a93de3ba9eb (DEFAULT_TABLE table=test-table [id=1db3c7f6caaf44a9bb4f4c5700cf7436]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:55.523326 1887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b72525b8424a9fa6cb1a93de3ba9eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:55.527305 2105 tablet_bootstrap.cc:492] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc: Bootstrap starting.
I20260926 20:40:55.530086 2021 tablet_service.cc:1467] Processing CreateTablet for tablet 26b72525b8424a9fa6cb1a93de3ba9eb (DEFAULT_TABLE table=test-table [id=1db3c7f6caaf44a9bb4f4c5700cf7436]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:40:55.530804 2105 tablet_bootstrap.cc:654] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:55.531754 2021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b72525b8424a9fa6cb1a93de3ba9eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:40:55.532410 2105 log.cc:824] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:55.535519 2105 tablet_bootstrap.cc:492] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc: No bootstrap required, opened a new log
I20260926 20:40:55.535856 2105 ts_tablet_manager.cc:1397] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260926 20:40:55.537627 2107 tablet_bootstrap.cc:492] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913: Bootstrap starting.
I20260926 20:40:55.541802 2107 tablet_bootstrap.cc:654] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:55.543323 2107 log.cc:824] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:55.546017 2109 tablet_bootstrap.cc:492] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434: Bootstrap starting.
I20260926 20:40:55.546852 2107 tablet_bootstrap.cc:492] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913: No bootstrap required, opened a new log
I20260926 20:40:55.547251 2107 ts_tablet_manager.cc:1397] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260926 20:40:55.547358 2105 raft_consensus.cc:348] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "501b9a732c99493b9bc0cab8c861b434" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45363 } } peers { permanent_uuid: "55b1de103c914f4c849343715aa0d0bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36495 } } peers { permanent_uuid: "8dacd89ad02b436ab74fb7314834e913" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36723 } }
I20260926 20:40:55.547901 2105 raft_consensus.cc:374] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:55.548049 2105 raft_consensus.cc:729] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55b1de103c914f4c849343715aa0d0bc, State: Initialized, Role: FOLLOWER
I20260926 20:40:55.548800 2105 consensus_queue.cc:260] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "501b9a732c99493b9bc0cab8c861b434" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45363 } } peers { permanent_uuid: "55b1de103c914f4c849343715aa0d0bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36495 } } peers { permanent_uuid: "8dacd89ad02b436ab74fb7314834e913" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36723 } }
I20260926 20:40:55.551123 2105 ts_tablet_manager.cc:1428] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260926 20:40:55.552229 2109 tablet_bootstrap.cc:654] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434: Neither blocks nor log segments found. Creating new log.
I20260926 20:40:55.553562 2109 log.cc:824] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434: Log is configured to *not* fsync() on all Append() calls
I20260926 20:40:55.556874 2109 tablet_bootstrap.cc:492] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434: No bootstrap required, opened a new log
I20260926 20:40:55.557200 2109 ts_tablet_manager.cc:1397] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260926 20:40:55.560053 2107 raft_consensus.cc:348] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "501b9a732c99493b9bc0cab8c861b434" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45363 } } peers { permanent_uuid: "55b1de103c914f4c849343715aa0d0bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36495 } } peers { permanent_uuid: "8dacd89ad02b436ab74fb7314834e913" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36723 } }
I20260926 20:40:55.560523 2107 raft_consensus.cc:374] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:55.560693 2107 raft_consensus.cc:729] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dacd89ad02b436ab74fb7314834e913, State: Initialized, Role: FOLLOWER
I20260926 20:40:55.561367 2107 consensus_queue.cc:260] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "501b9a732c99493b9bc0cab8c861b434" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45363 } } peers { permanent_uuid: "55b1de103c914f4c849343715aa0d0bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36495 } } peers { permanent_uuid: "8dacd89ad02b436ab74fb7314834e913" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36723 } }
I20260926 20:40:55.562642 1952 heartbeater.cc:503] Master 127.23.44.254:42809 was elected leader, sending a full tablet report...
I20260926 20:40:55.563258 2107 ts_tablet_manager.cc:1428] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260926 20:40:55.565752 2109 raft_consensus.cc:348] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "501b9a732c99493b9bc0cab8c861b434" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45363 } } peers { permanent_uuid: "55b1de103c914f4c849343715aa0d0bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36495 } } peers { permanent_uuid: "8dacd89ad02b436ab74fb7314834e913" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36723 } }
I20260926 20:40:55.566233 2109 raft_consensus.cc:374] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:40:55.566349 2109 raft_consensus.cc:729] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 501b9a732c99493b9bc0cab8c861b434, State: Initialized, Role: FOLLOWER
I20260926 20:40:55.567337 2109 consensus_queue.cc:260] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "501b9a732c99493b9bc0cab8c861b434" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45363 } } peers { permanent_uuid: "55b1de103c914f4c849343715aa0d0bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36495 } } peers { permanent_uuid: "8dacd89ad02b436ab74fb7314834e913" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36723 } }
I20260926 20:40:55.567682 2113 raft_consensus.cc:482] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:40:55.568002 2113 raft_consensus.cc:504] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "501b9a732c99493b9bc0cab8c861b434" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45363 } } peers { permanent_uuid: "55b1de103c914f4c849343715aa0d0bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36495 } } peers { permanent_uuid: "8dacd89ad02b436ab74fb7314834e913" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36723 } }
I20260926 20:40:55.568533 2086 heartbeater.cc:503] Master 127.23.44.254:42809 was elected leader, sending a full tablet report...
W20260926 20:40:55.570024 1819 tablet.cc:2369] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:40:55.570861 2109 ts_tablet_manager.cc:1428] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434: Time spent starting tablet: real 0.014s user 0.011s sys 0.000s
I20260926 20:40:55.571619 2113 leader_election.cc:288] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55b1de103c914f4c849343715aa0d0bc (127.23.44.193:36495), 8dacd89ad02b436ab74fb7314834e913 (127.23.44.194:36723)
I20260926 20:40:55.577869 2101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.571346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:58049 (local address 127.23.44.193:36495)
0926 20:40:55.571672 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:55.571679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:55.571701 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:40:55.571771 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:55.571776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:55.571867 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:55.571986 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:55.572451 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.573628 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.575639 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.576930 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.577079 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.577134 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:55.577518 (+ 384us) server_negotiation.cc:299] Negotiation successful
0926 20:40:55.577657 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260926 20:40:55.577907 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.570573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:36495 (local address 127.23.44.195:58049)
0926 20:40:55.571277 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:55.571297 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:55.571410 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:55.572029 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:55.572034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:55.572063 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:55.572271 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:55.572289 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.573786 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.573791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:55.575464 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:55.575475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.577081 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.577087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:55.577308 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.577378 (+ 70us) client_negotiation.cc:769] Sending connection context
0926 20:40:55.577490 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 20:40:55.577628 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556}
I20260926 20:40:55.579161 2115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.571834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:36723 (local address 127.23.44.195:48561)
0926 20:40:55.572706 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0926 20:40:55.572723 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:40:55.572868 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:40:55.573200 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:40:55.573206 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:40:55.573231 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:40:55.573520 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:55.573528 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.574909 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.574913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:55.576814 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:40:55.576829 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.578527 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.578533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:40:55.578701 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.578737 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 20:40:55.578816 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 20:40:55.578904 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":668,"threads_started":1}
I20260926 20:40:55.579248 2104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.572036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:48561 (local address 127.23.44.194:36723)
0926 20:40:55.572235 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:55.572241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:55.572825 (+ 584us) server_negotiation.cc:407] Connection header received
0926 20:40:55.572996 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:55.573001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:55.573076 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:55.573164 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:40:55.573662 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.574725 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.576976 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.578331 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.578444 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.578550 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:55.578872 (+ 322us) server_negotiation.cc:299] Negotiation successful
0926 20:40:55.579002 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:40:55.579164 1773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b72525b8424a9fa6cb1a93de3ba9eb" candidate_uuid: "501b9a732c99493b9bc0cab8c861b434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55b1de103c914f4c849343715aa0d0bc" is_pre_election: true
I20260926 20:40:55.579779 1773 raft_consensus.cc:2454] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 501b9a732c99493b9bc0cab8c861b434 in term 0.
I20260926 20:40:55.580055 1907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b72525b8424a9fa6cb1a93de3ba9eb" candidate_uuid: "501b9a732c99493b9bc0cab8c861b434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dacd89ad02b436ab74fb7314834e913" is_pre_election: true
I20260926 20:40:55.580474 1975 leader_election.cc:302] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 501b9a732c99493b9bc0cab8c861b434, 55b1de103c914f4c849343715aa0d0bc; no voters:
I20260926 20:40:55.580662 1907 raft_consensus.cc:2454] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 501b9a732c99493b9bc0cab8c861b434 in term 0.
I20260926 20:40:55.580919 2113 raft_consensus.cc:2781] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:40:55.581063 2113 raft_consensus.cc:482] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:40:55.581185 2113 raft_consensus.cc:3037] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:55.583375 2113 raft_consensus.cc:504] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "501b9a732c99493b9bc0cab8c861b434" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45363 } } peers { permanent_uuid: "55b1de103c914f4c849343715aa0d0bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36495 } } peers { permanent_uuid: "8dacd89ad02b436ab74fb7314834e913" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36723 } }
I20260926 20:40:55.584012 2113 leader_election.cc:288] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [CANDIDATE]: Term 1 election: Requested vote from peers 55b1de103c914f4c849343715aa0d0bc (127.23.44.193:36495), 8dacd89ad02b436ab74fb7314834e913 (127.23.44.194:36723)
I20260926 20:40:55.584757 1773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b72525b8424a9fa6cb1a93de3ba9eb" candidate_uuid: "501b9a732c99493b9bc0cab8c861b434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55b1de103c914f4c849343715aa0d0bc"
I20260926 20:40:55.584981 1773 raft_consensus.cc:3037] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:55.584893 1907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b72525b8424a9fa6cb1a93de3ba9eb" candidate_uuid: "501b9a732c99493b9bc0cab8c861b434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dacd89ad02b436ab74fb7314834e913"
I20260926 20:40:55.585187 1907 raft_consensus.cc:3037] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:40:55.587266 1773 raft_consensus.cc:2454] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 501b9a732c99493b9bc0cab8c861b434 in term 1.
I20260926 20:40:55.587757 1975 leader_election.cc:302] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 501b9a732c99493b9bc0cab8c861b434, 55b1de103c914f4c849343715aa0d0bc; no voters:
I20260926 20:40:55.587910 1907 raft_consensus.cc:2454] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 501b9a732c99493b9bc0cab8c861b434 in term 1.
I20260926 20:40:55.588069 2113 raft_consensus.cc:2781] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:40:55.588255 2113 raft_consensus.cc:686] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [term 1 LEADER]: Becoming Leader. State: Replica: 501b9a732c99493b9bc0cab8c861b434, State: Running, Role: LEADER
I20260926 20:40:55.588901 2113 consensus_queue.cc:237] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "501b9a732c99493b9bc0cab8c861b434" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45363 } } peers { permanent_uuid: "55b1de103c914f4c849343715aa0d0bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36495 } } peers { permanent_uuid: "8dacd89ad02b436ab74fb7314834e913" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36723 } }
I20260926 20:40:55.594254 1629 catalog_manager.cc:5495] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 reported cstate change: term changed from 0 to 1, leader changed from <none> to 501b9a732c99493b9bc0cab8c861b434 (127.23.44.195). New cstate: current_term: 1 leader_uuid: "501b9a732c99493b9bc0cab8c861b434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "501b9a732c99493b9bc0cab8c861b434" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55b1de103c914f4c849343715aa0d0bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dacd89ad02b436ab74fb7314834e913" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36723 } health_report { overall_health: UNKNOWN } } }
W20260926 20:40:55.603122 1953 tablet.cc:2369] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:40:55.623652 2102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.617109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.23.44.195:45363)
0926 20:40:55.617376 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:55.617382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:55.617547 (+ 165us) server_negotiation.cc:407] Connection header received
0926 20:40:55.617687 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:55.617692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:55.617799 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:55.617952 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:40:55.618331 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.619042 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.620934 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.621372 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.621472 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.622258 (+ 786us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:40:55.622974 (+ 716us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:40:55.623149 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:55.623325 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 20:40:55.623415 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260926 20:40:55.632581 1773 raft_consensus.cc:1261] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc [term 1 FOLLOWER]: Refusing update from remote peer 501b9a732c99493b9bc0cab8c861b434: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:55.632582 1907 raft_consensus.cc:1261] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913 [term 1 FOLLOWER]: Refusing update from remote peer 501b9a732c99493b9bc0cab8c861b434: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:40:55.633900 2113 consensus_queue.cc:1035] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55b1de103c914f4c849343715aa0d0bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:55.634234 2114 consensus_queue.cc:1035] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dacd89ad02b436ab74fb7314834e913" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:40:55.647141 2118 mvcc.cc:205] Tried to move back new op lower bound from 7333704727059963904 to 7333704726896726016. Current Snapshot: MvccSnapshot[applied={T|T < 7333704727059963904}]
I20260926 20:40:55.659886 2120 mvcc.cc:205] Tried to move back new op lower bound from 7333704727059963904 to 7333704726896726016. Current Snapshot: MvccSnapshot[applied={T|T < 7333704727059963904}]
I20260926 20:40:55.677614 1628 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:55.677983 1628 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:40:55.682442 1628 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55e8d7484d314ed5978d95c55bf67074: Sending DeleteTablet for 3 replicas of tablet 26b72525b8424a9fa6cb1a93de3ba9eb
I20260926 20:40:55.683645 2021 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b72525b8424a9fa6cb1a93de3ba9eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:55 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:42738
I20260926 20:40:55.683656 1753 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b72525b8424a9fa6cb1a93de3ba9eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:55 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:35604
I20260926 20:40:55.684803 2129 tablet_replica.cc:330] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc: stopping tablet replica
I20260926 20:40:55.685004 1887 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b72525b8424a9fa6cb1a93de3ba9eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:40:55 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:52240
I20260926 20:40:55.685333 2129 raft_consensus.cc:2229] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:55.685863 2129 raft_consensus.cc:2258] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:55.685971 2130 tablet_replica.cc:330] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913: stopping tablet replica
I20260926 20:40:55.686802 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 1690
I20260926 20:40:55.686493 2130 raft_consensus.cc:2229] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:40:55.687614 2130 raft_consensus.cc:2258] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:55.688367 2129 ts_tablet_manager.cc:1905] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:55.693115 2128 tablet_replica.cc:330] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434: stopping tablet replica
I20260926 20:40:55.693634 2128 raft_consensus.cc:2229] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:40:55.694480 2128 raft_consensus.cc:2258] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:40:55.696691 2128 ts_tablet_manager.cc:1905] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:55.697573 2129 ts_tablet_manager.cc:1918] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:55.697814 2129 log.cc:1196] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/26b72525b8424a9fa6cb1a93de3ba9eb
I20260926 20:40:55.698446 2129 ts_tablet_manager.cc:1939] T 26b72525b8424a9fa6cb1a93de3ba9eb P 55b1de103c914f4c849343715aa0d0bc: Deleting consensus metadata
I20260926 20:40:55.698642 2130 ts_tablet_manager.cc:1905] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:40:55.700506 1614 catalog_manager.cc:4841] TS 55b1de103c914f4c849343715aa0d0bc (127.23.44.193:36495): tablet 26b72525b8424a9fa6cb1a93de3ba9eb (table test-table [id=1db3c7f6caaf44a9bb4f4c5700cf7436]) successfully deleted
I20260926 20:40:55.704573 2128 ts_tablet_manager.cc:1918] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:55.704911 2128 log.cc:1196] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/26b72525b8424a9fa6cb1a93de3ba9eb
I20260926 20:40:55.705569 2128 ts_tablet_manager.cc:1939] T 26b72525b8424a9fa6cb1a93de3ba9eb P 501b9a732c99493b9bc0cab8c861b434: Deleting consensus metadata
I20260926 20:40:55.707479 1613 catalog_manager.cc:4841] TS 501b9a732c99493b9bc0cab8c861b434 (127.23.44.195:45363): tablet 26b72525b8424a9fa6cb1a93de3ba9eb (table test-table [id=1db3c7f6caaf44a9bb4f4c5700cf7436]) successfully deleted
I20260926 20:40:55.708101 2130 ts_tablet_manager.cc:1918] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:40:55.708288 2130 log.cc:1196] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/26b72525b8424a9fa6cb1a93de3ba9eb
I20260926 20:40:55.708879 2130 ts_tablet_manager.cc:1939] T 26b72525b8424a9fa6cb1a93de3ba9eb P 8dacd89ad02b436ab74fb7314834e913: Deleting consensus metadata
I20260926 20:40:55.710731 1616 catalog_manager.cc:4841] TS 8dacd89ad02b436ab74fb7314834e913 (127.23.44.194:36723): tablet 26b72525b8424a9fa6cb1a93de3ba9eb (table test-table [id=1db3c7f6caaf44a9bb4f4c5700cf7436]) successfully deleted
Sep 26 20:40:55 dist-test-slave-4d5t krb5kdc[1584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:40:55.721836 2101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.690650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.23.44.193:36495)
0926 20:40:55.690832 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:55.690838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:55.703386 (+ 12548us) server_negotiation.cc:407] Connection header received
0926 20:40:55.703642 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:55.703648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:55.703745 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:55.703853 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:55.705756 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.707056 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.709177 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.709872 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.710022 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.714547 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:55.714570 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:55.714590 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:55.714646 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:55.718648 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:55.719526 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:55.719537 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:55.719541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:55.719709 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:55.720112 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:55.720117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:55.720121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:55.720622 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:55.720939 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:55.721191 (+ 252us) server_negotiation.cc:299] Negotiation successful
0926 20:40:55.721417 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 20:40:55.722707 1813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:55.817910 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 1690
I20260926 20:40:55.841086 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 1824
Sep 26 20:40:55 dist-test-slave-4d5t krb5kdc[1584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:40:55.861354 2104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:55.841997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52248 (local address 127.23.44.194:36723)
0926 20:40:55.842166 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:55.842170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:55.842189 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:40:55.842316 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:55.842320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:55.842386 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:55.842472 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:55.844111 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.845093 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.846991 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:55.847446 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:55.847572 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:55.853506 (+ 5934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:55.853518 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:55.853535 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:55.853589 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:55.857660 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:55.858942 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:55.858953 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:55.858957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:55.859119 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:55.859555 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:55.859560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:55.859564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:55.860087 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:55.860388 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:55.860721 (+ 333us) server_negotiation.cc:299] Negotiation successful
0926 20:40:55.860953 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 20:40:55.862162 1947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:55.987793 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 1824
I20260926 20:40:56.004128 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 1958
Sep 26 20:40:56 dist-test-slave-4d5t krb5kdc[1584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:40:56.025228 2102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:40:56.005176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42764 (local address 127.23.44.195:45363)
0926 20:40:56.005445 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0926 20:40:56.005452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:40:56.005484 (+ 32us) server_negotiation.cc:407] Connection header received
0926 20:40:56.005645 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:40:56.005652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:40:56.005774 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:40:56.005961 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:40:56.007668 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:56.008932 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:56.010376 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:40:56.011105 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:40:56.011262 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:40:56.017035 (+ 5773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:40:56.017052 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:40:56.017085 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:40:56.017150 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:40:56.021569 (+ 4419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:56.022613 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:56.022625 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:56.022629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:56.022849 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:40:56.023325 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:40:56.023331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:40:56.023335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:40:56.023907 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:40:56.024270 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0926 20:40:56.024500 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 20:40:56.024776 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260926 20:40:56.026505 2081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:56.156668 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 1958
I20260926 20:40:56.176450 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 1597
I20260926 20:40:56.177817 1659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:40:56.285420 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 1597
2026-09-26T20:40:56Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4531 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:40:56 dist-test-slave-4d5t krb5kdc[2138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:40:56 dist-test-slave-4d5t krb5kdc[2138](info): set up 2 sockets
Sep 26 20:40:56 dist-test-slave-4d5t krb5kdc[2138](info): commencing operation
krb5kdc: starting...
W20260926 20:40:57.462738 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.002s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:40:57 dist-test-slave-4d5t krb5kdc[2138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:40:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:40:57Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/rangeradmin_127.23.44.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/rangeradmin_127.23.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/rangerlookup_127.23.44.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/rangerlookup_127.23.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/HTTP_127.23.44.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/HTTP_127.23.44.212@KRBTEST.COM.keytab.
I20260926 20:40:57.836719 23731 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskO4QBsv/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/postgres -l logfile start
2026-09-26 20:41:00.626 UTC [2168] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 20:41:00.627 UTC [2168] LOG: listening on IPv4 address "127.23.44.212", port 37057
2026-09-26 20:41:00.631 UTC [2168] LOG: listening on Unix socket "/tmp/.s.PGSQL.37057"
2026-09-26 20:41:00.652 UTC [2171] LOG: database system was shut down at 2026-09-26 20:40:58 UTC
2026-09-26 20:41:00.656 UTC [2168] LOG: database system is ready to accept connections
I20260926 20:41:01.657960 23731 mini_postgres.cc:96] Postgres bound to 37057
2026-09-26 20:41:01.678 UTC [2179] FATAL: database "slave" does not exist
127.23.44.212:37057 - accepting connections
I20260926 20:41:01.678901 23731 mini_ranger.cc:161] Starting Ranger...
I20260926 20:41:01.708254 23731 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 20:41:02.237046 23731 mini_ranger.cc:87] Created ranger Postgres database
I20260926 20:41:02.237206 23731 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-admin
2026-09-26 20:41:02,548 [I] DB FLAVOR :POSTGRES
2026-09-26 20:41:02,549 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 20:41:02,549 [I] Checking connection..
2026-09-26 20:41:02,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 20:41:02,846 [I] Checking connection passed.
2026-09-26 20:41:02,846 [I] --------- Verifying version history table ---------
2026-09-26 20:41:02,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 20:41:03,110 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 20:41:03,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 20:41:03,384 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 20:41:03,385 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 20:41:03,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 20:41:03.647 UTC [2244] WARNING: there is no transaction in progress
2026-09-26 20:41:03,654 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 20:41:03,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 20:41:03,923 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 20:41:03,924 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 20:41:03,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 20:41:04,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 20:41:04,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4d5t.c.gcp-upstream.internal','N') ;"
2026-09-26 20:41:04,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 20:41:05,000 [I] Table x_portal_user does not exist in database ranger
2026-09-26 20:41:05,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 20:41:05,286 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 20:41:05,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 20:41:05,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 20:41:05,853 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 20:41:05,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 20:41:06.956 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:06.974 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:06.993 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07.010 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07.030 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07.054 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07.127 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07.136 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07.152 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07.164 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07.175 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07.190 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07.200 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07.207 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07.213 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07.221 UTC [2352] WARNING: there is no transaction in progress
2026-09-26 20:41:07,955 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 20:41:07,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 20:41:08,200 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 20:41:08,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 20:41:08,486 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 20:41:08,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 20:41:08,774 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 20:41:08,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 20:41:09,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 20:41:09,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4d5t.c.gcp-upstream.internal';"
2026-09-26 20:41:09,573 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 20:41:09,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 20:41:09,848 [I] DB_PATCHES have already been applied
I20260926 20:41:09.854671 23731 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-admin:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskO4QBsv/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 20:41:09.854777 23731 mini_ranger.cc:193] Using host: 127.23.44.212
I20260926 20:41:09.871001 23731 mini_ranger.cc:227] Ranger admin URL: http://127.23.44.212:54913
Sep 26, 2026 8:41:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 8:41:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 8:41:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 8:41:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 8:41:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 8:41:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 8:41:10 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 20:41:11.081888 23731 mini_ranger.cc:160] Time spent starting Ranger: real 9.403s user 0.000s sys 0.045s
Sep 26, 2026 8:41:32 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 8:41:32 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 8:41:32 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 8:41:32 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 8:41:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 8:41:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 8:41:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 8:41:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 8:41:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 8:41:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 8:41:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 8:41:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 8:41:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 8:41:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 8:41:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 8:41:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 8:41:32 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 8:41:33 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 20:41:34.061138 23731 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/rangerkms_127.23.44.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/rangerkms_127.23.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.44.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.44.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.44.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/HTTP_127.23.44.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.44.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/HTTP_127.23.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 20:41:34 dist-test-slave-4d5t krb5kdc[2138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455294, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 20:41:34.240255 23731 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 20:41:34.263247 23731 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 20:41:35.244509 23731 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 20:41:35.244777 23731 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-kms
I20260926 20:41:35.244922 23731 mini_ranger_kms.cc:227] Using postgres at 127.23.44.212:37057
2026-09-26 20:41:35,702 [I] DB FLAVOR :POSTGRES
2026-09-26 20:41:35,703 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 20:41:35,703 [I] Checking connection
2026-09-26 20:41:35,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 20:41:35,969 [I] connection success
2026-09-26 20:41:35,969 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 20:41:35,969 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 20:41:35,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 20:41:36,238 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 20:41:36,239 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 20:41:36,239 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 20:41:36,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:37057/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 20:41:36,530 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 20:41:37.136340 23731 mini_ranger_kms.cc:310] Created kms service
I20260926 20:41:37.367133 23731 mini_ranger_kms.cc:326] Created kudu user
I20260926 20:41:37.418279 23731 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 20:41:37 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455294, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:41:37.883193 23731 mini_ranger_kms.cc:384] Added ranger policy
I20260926 20:41:37.883524 23731 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-kms:/tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 20:41:37.883574 23731 mini_ranger_kms.cc:240] Using host: 127.23.44.212
I20260926 20:41:37.902194 23731 mini_ranger_kms.cc:276] Ranger KMS PID: 2533
I20260926 20:41:37.902419 23731 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.44.212:36869
Sep 26, 2026 8:41:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 8:41:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 8:41:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 8:41:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 8:41:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 8:41:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 20:41:40.292022 23731 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.052s user 0.100s sys 0.376s
I20260926 20:41:40.292281 23731 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 20:41:40.292505 23731 mini_ranger_kms.cc:401] 127.23.44.212:36869/kms/v1/keys
Sep 26 20:41:40 dist-test-slave-4d5t krb5kdc[2138](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790455300, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.44.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 20:41:41 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455300, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.44.212@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
Sep 26 20:41:41 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455300, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.44.212@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
Sep 26 20:41:43 dist-test-slave-4d5t krb5kdc[2138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:41:43.785233 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:35617
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:35617
--ranger_config_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.44.212:36869/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:41:44.344683 2583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:41:44.345016 2583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:41:44.345134 2583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:41:44.353935 2583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:41:44.354092 2583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:41:44.354158 2583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:41:44.354239 2583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:41:44.354306 2583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:41:44.366866 2583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.44.212:36869/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:35617
--ranger_config_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:35617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:41:44.368328 2583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:41:44.369755 2583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:41:44.382874 2588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:41:44.383029 2589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:41:44.383901 2591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:41:44.384505 2583 server_base.cc:1034] running on GCE node
I20260926 20:41:44.385430 2583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:41:44.387003 2583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:41:44.388273 2583 hybrid_clock.cc:648] HybridClock initialized: now 1790455304388192 us; error 34 us; skew 500 ppm
I20260926 20:41:44.388806 2583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:41:44 dist-test-slave-4d5t krb5kdc[2138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455304, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:41:44.392735 2583 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:41:44.395134 2583 webserver.cc:460] Webserver started at http://127.23.44.254:33827/ using document root <none> and password file <none>
I20260926 20:41:44.395875 2583 fs_manager.cc:359] Metadata directory not provided
I20260926 20:41:44.396046 2583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:41:44.396384 2583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:41:44 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455304, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:41:44.467068 2583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "fe912c4c103a48c0a37e65f46c602f2f"
format_stamp: "Formatted at 2026-09-26 20:41:44 on dist-test-slave-4d5t"
server_key: "3087e2244a64ce24ffb7ef3e19c948cd"
server_key_iv: "d0cce574612a178320bee221a81b9990"
server_key_version: "kuduclusterkey@0"
I20260926 20:41:44.467994 2583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "fe912c4c103a48c0a37e65f46c602f2f"
format_stamp: "Formatted at 2026-09-26 20:41:44 on dist-test-slave-4d5t"
server_key: "3087e2244a64ce24ffb7ef3e19c948cd"
server_key_iv: "d0cce574612a178320bee221a81b9990"
server_key_version: "kuduclusterkey@0"
I20260926 20:41:44.474747 2583 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260926 20:41:44.497637 2598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:41:44.499702 2583 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.004s
I20260926 20:41:44.500097 2583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "fe912c4c103a48c0a37e65f46c602f2f"
format_stamp: "Formatted at 2026-09-26 20:41:44 on dist-test-slave-4d5t"
server_key: "3087e2244a64ce24ffb7ef3e19c948cd"
server_key_iv: "d0cce574612a178320bee221a81b9990"
server_key_version: "kuduclusterkey@0"
I20260926 20:41:44.500408 2583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:41:44.517640 2583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:41:44.522533 2583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:41:44.522923 2583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:41:44.555655 2583 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:35617
I20260926 20:41:44.555694 2649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:35617 every 8 connection(s)
I20260926 20:41:44.557425 2583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:41:44.561473 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 2583
I20260926 20:41:44.561687 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:41:44.562381 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:41:44.562604 23731 external_mini_cluster.cc:1428] Setting key 1aadc80e604ee40ed59dc51433e362e7
I20260926 20:41:44.586790 2650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f: Bootstrap starting.
I20260926 20:41:44.591449 2650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f: Neither blocks nor log segments found. Creating new log.
I20260926 20:41:44.592911 2650 log.cc:824] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f: Log is configured to *not* fsync() on all Append() calls
I20260926 20:41:44.597039 2650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f: No bootstrap required, opened a new log
Sep 26 20:41:44 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:41:44.607004 2653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:44.565115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54412 (local address 127.23.44.254:35617)
0926 20:41:44.581260 (+ 16145us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:44.581279 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:44.581370 (+ 91us) server_negotiation.cc:407] Connection header received
0926 20:41:44.584776 (+ 3406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:44.584817 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:44.585494 (+ 677us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:44.586031 (+ 537us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:41:44.588278 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:44.590718 (+ 2440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:44.592644 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:44.593933 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:44.594157 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:44.598270 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:41:44.598279 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:41:44.598296 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:41:44.598344 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:41:44.602104 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:44.602867 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:44.602879 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:44.602886 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:44.603047 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:44.603461 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:44.603468 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:44.603475 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:44.604222 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:41:44.604607 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:44.605180 (+ 573us) server_negotiation.cc:299] Negotiation successful
0926 20:41:44.605857 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15816,"thread_start_us":15489,"threads_started":1}
I20260926 20:41:44.610422 2650 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe912c4c103a48c0a37e65f46c602f2f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35617 } }
I20260926 20:41:44.610965 2650 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:41:44.611091 2650 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe912c4c103a48c0a37e65f46c602f2f, State: Initialized, Role: FOLLOWER
I20260926 20:41:44.611770 2650 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe912c4c103a48c0a37e65f46c602f2f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35617 } }
I20260926 20:41:44.612110 2650 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:41:44.612282 2650 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:41:44.612500 2650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:41:44.616338 2650 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe912c4c103a48c0a37e65f46c602f2f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35617 } }
I20260926 20:41:44.617064 2650 leader_election.cc:302] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe912c4c103a48c0a37e65f46c602f2f; no voters:
I20260926 20:41:44.618130 2650 leader_election.cc:288] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:41:44.618219 2655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:41:44.621243 2650 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:41:44.625669 2655 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [term 1 LEADER]: Becoming Leader. State: Replica: fe912c4c103a48c0a37e65f46c602f2f, State: Running, Role: LEADER
I20260926 20:41:44.626274 2655 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe912c4c103a48c0a37e65f46c602f2f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35617 } }
I20260926 20:41:44.631187 2657 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe912c4c103a48c0a37e65f46c602f2f. Latest consensus state: current_term: 1 leader_uuid: "fe912c4c103a48c0a37e65f46c602f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe912c4c103a48c0a37e65f46c602f2f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35617 } } }
I20260926 20:41:44.631105 2656 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe912c4c103a48c0a37e65f46c602f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe912c4c103a48c0a37e65f46c602f2f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35617 } } }
I20260926 20:41:44.631924 2657 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [sys.catalog]: This master's current role is: LEADER
I20260926 20:41:44.631924 2656 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f [sys.catalog]: This master's current role is: LEADER
I20260926 20:41:44.650089 2662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:41:44.658361 2662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:41:44.659328 2650 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 20:41:44.681058 2662 catalog_manager.cc:1347] Generated new cluster ID: 5d19887f80b04ea6806aa7db41d02e76
I20260926 20:41:44.681246 2662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:41:44.691115 2662 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:41:44.692967 2662 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:41:44.702775 2662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f: Generated new TSK 0
I20260926 20:41:44.703806 2662 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:41:45.529778 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.44.212:36869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:35617
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:41:45 dist-test-slave-4d5t krb5kdc[2138](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790455305, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 20:41:45.995141 2695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:41:45.995586 2695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:41:45.995785 2695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:41:46.009665 2695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:41:46.009866 2695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:41:46.010139 2695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:41:46.029075 2695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.44.212:36869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:35617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:41:46.031203 2695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:41:46.033253 2695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:41:46.070612 2706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:41:46.071805 2705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:41:46.075327 2708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:41:46.080904 2695 server_base.cc:1034] running on GCE node
I20260926 20:41:46.081893 2695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:41:46.086763 2695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:41:46.088308 2695 hybrid_clock.cc:648] HybridClock initialized: now 1790455306088196 us; error 123 us; skew 500 ppm
I20260926 20:41:46.088914 2695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:41:46 dist-test-slave-4d5t krb5kdc[2138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455306, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:41:46.093600 2695 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:41:46.124720 2695 webserver.cc:460] Webserver started at http://127.23.44.193:32841/ using document root <none> and password file <none>
I20260926 20:41:46.126080 2695 fs_manager.cc:359] Metadata directory not provided
I20260926 20:41:46.126348 2695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:41:46.126878 2695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:41:46 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455306, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:41:46.170987 2695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "e70d89cb65c74999aaa3cf57c27b62d4"
format_stamp: "Formatted at 2026-09-26 20:41:46 on dist-test-slave-4d5t"
server_key: "48df076d9cbe392a795bd5939a79e8f6"
server_key_iv: "8ef0670234fc62e7d4e6d09df92ad8a3"
server_key_version: "kuduclusterkey@0"
I20260926 20:41:46.174952 2695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "e70d89cb65c74999aaa3cf57c27b62d4"
format_stamp: "Formatted at 2026-09-26 20:41:46 on dist-test-slave-4d5t"
server_key: "48df076d9cbe392a795bd5939a79e8f6"
server_key_iv: "8ef0670234fc62e7d4e6d09df92ad8a3"
server_key_version: "kuduclusterkey@0"
I20260926 20:41:46.191483 2695 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.006s sys 0.000s
I20260926 20:41:46.230427 2715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:41:46.235960 2695 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.003s sys 0.002s
I20260926 20:41:46.236204 2695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "e70d89cb65c74999aaa3cf57c27b62d4"
format_stamp: "Formatted at 2026-09-26 20:41:46 on dist-test-slave-4d5t"
server_key: "48df076d9cbe392a795bd5939a79e8f6"
server_key_iv: "8ef0670234fc62e7d4e6d09df92ad8a3"
server_key_version: "kuduclusterkey@0"
I20260926 20:41:46.236483 2695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:41:46.275152 2695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:41:46.285367 2695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:41:46.285817 2695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:41:46.316864 2695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:41:46.325791 2695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:41:46.325991 2695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 20:41:46.326191 2695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:41:46.326337 2695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:41:46 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455305, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:41:46.574066 2827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:36493 every 8 connection(s)
I20260926 20:41:46.574465 2695 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:36493
I20260926 20:41:46.576362 2695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:41:46.581642 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 2695
I20260926 20:41:46.581903 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:41:46.582957 23731 external_mini_cluster.cc:1428] Setting key 62f52d47b69413005371ffb9b053c2dc
Sep 26 20:41:46 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455306, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:41:46.619326 2831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:46.589261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:42069 (local address 127.23.44.254:35617)
0926 20:41:46.594957 (+ 5696us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:46.594963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:46.594984 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:41:46.595048 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:46.595053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:46.595137 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:46.595244 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:41:46.597502 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:46.598699 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:46.602784 (+ 4085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:46.603376 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:46.603477 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:46.608969 (+ 5492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:41:46.608979 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:41:46.608987 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:41:46.609046 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:41:46.613369 (+ 4323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:46.614559 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:46.614566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:46.614571 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:46.614743 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:46.615351 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:46.615357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:46.615361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:46.615771 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:41:46.615918 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:46.618603 (+ 2685us) server_negotiation.cc:299] Negotiation successful
0926 20:41:46.618891 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5536,"thread_start_us":10376,"threads_started":1}
I20260926 20:41:46.622335 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:46.588590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:35617 (local address 127.23.44.193:42069)
0926 20:41:46.591686 (+ 3096us) negotiation.cc:107] Waiting for socket to connect
0926 20:41:46.591736 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 20:41:46.594193 (+ 2457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:41:46.595711 (+ 1518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:41:46.595724 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:41:46.596380 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:41:46.597264 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:46.597291 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:46.598919 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:46.598924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:46.602542 (+ 3618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:46.602564 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:46.603836 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:46.603843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:46.604330 (+ 487us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:46.605312 (+ 982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:41:46.605347 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:41:46.608598 (+ 3251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:41:46.613637 (+ 5039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:41:46.613649 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:41:46.613671 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:41:46.614391 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:41:46.614909 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:41:46.614914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:41:46.614919 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:41:46.615205 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:41:46.616029 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:41:46.616039 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:41:46.616541 (+ 502us) client_negotiation.cc:769] Sending connection context
0926 20:41:46.619946 (+ 3405us) client_negotiation.cc:240] Negotiation successful
0926 20:41:46.620642 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2764,"thread_start_us":2520,"threads_started":1}
I20260926 20:41:46.624641 2828 heartbeater.cc:347] Connected to a master server at 127.23.44.254:35617
I20260926 20:41:46.625005 2828 heartbeater.cc:464] Registering TS with master...
I20260926 20:41:46.625914 2828 heartbeater.cc:511] Master 127.23.44.254:35617 requested a full tablet report, sending...
I20260926 20:41:46.628578 2615 ts_manager.cc:194] Registered new tserver with Master: e70d89cb65c74999aaa3cf57c27b62d4 (127.23.44.193:36493)
I20260926 20:41:46.630697 2615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:42069
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:41:46.678184 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.44.212:36869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:35617
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:41:46 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455305, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
W20260926 20:41:47.222929 2834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:41:47.223395 2834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:41:47.223575 2834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:41:47.236146 2834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:41:47.236336 2834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:41:47.236522 2834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:41:47.250139 2834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.44.212:36869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:35617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:41:47.251472 2834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:41:47.252795 2834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:41:47.267470 2851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:41:47.267508 2848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:41:47.267438 2849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:41:47.268302 2834 server_base.cc:1034] running on GCE node
I20260926 20:41:47.269097 2834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:41:47.270244 2834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:41:47.271525 2834 hybrid_clock.cc:648] HybridClock initialized: now 1790455307271458 us; error 55 us; skew 500 ppm
I20260926 20:41:47.271921 2834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:41:47 dist-test-slave-4d5t krb5kdc[2138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455307, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:41:47.275441 2834 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:41:47.277194 2834 webserver.cc:460] Webserver started at http://127.23.44.194:35119/ using document root <none> and password file <none>
I20260926 20:41:47.277884 2834 fs_manager.cc:359] Metadata directory not provided
I20260926 20:41:47.277990 2834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:41:47.278247 2834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:41:47 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455307, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:41:47.296948 2834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "e479e166f72c419fb82d306a08c10694"
format_stamp: "Formatted at 2026-09-26 20:41:47 on dist-test-slave-4d5t"
server_key: "4d9a8508edaa7886faf6cf2f5d761898"
server_key_iv: "a35e28438912fb4540d45a9a5f1712f7"
server_key_version: "kuduclusterkey@0"
I20260926 20:41:47.297865 2834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "e479e166f72c419fb82d306a08c10694"
format_stamp: "Formatted at 2026-09-26 20:41:47 on dist-test-slave-4d5t"
server_key: "4d9a8508edaa7886faf6cf2f5d761898"
server_key_iv: "a35e28438912fb4540d45a9a5f1712f7"
server_key_version: "kuduclusterkey@0"
I20260926 20:41:47.304399 2834 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260926 20:41:47.322572 2858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:41:47.324280 2834 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260926 20:41:47.324493 2834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "e479e166f72c419fb82d306a08c10694"
format_stamp: "Formatted at 2026-09-26 20:41:47 on dist-test-slave-4d5t"
server_key: "4d9a8508edaa7886faf6cf2f5d761898"
server_key_iv: "a35e28438912fb4540d45a9a5f1712f7"
server_key_version: "kuduclusterkey@0"
I20260926 20:41:47.324797 2834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:41:47.349574 2834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:41:47.354861 2834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:41:47.355299 2834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:41:47.356616 2834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:41:47.358721 2834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:41:47.358892 2834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:41:47.359174 2834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:41:47.359311 2834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:41:47.405450 2834 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:41069
I20260926 20:41:47.405485 2970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:41069 every 8 connection(s)
I20260926 20:41:47.407127 2834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:41:47.413306 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 2834
I20260926 20:41:47.413501 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:41:47.413950 23731 external_mini_cluster.cc:1428] Setting key 67b0af22c78052acd0dce505775c32b2
Sep 26 20:41:47 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455307, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:41:47.431147 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:47.410371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:44395 (local address 127.23.44.254:35617)
0926 20:41:47.410990 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:47.410996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:47.411685 (+ 689us) server_negotiation.cc:407] Connection header received
0926 20:41:47.413567 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:47.413573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:47.413672 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:47.413804 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:41:47.415553 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:47.416974 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:47.418620 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:47.419560 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:47.419725 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:47.424195 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:41:47.424201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:41:47.424204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:41:47.424246 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:41:47.426994 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:47.427886 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:47.427892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:47.427895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:47.428026 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:47.428594 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:47.428602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:47.428608 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:47.429225 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:41:47.429409 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:47.430350 (+ 941us) server_negotiation.cc:299] Negotiation successful
0926 20:41:47.430640 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":332,"threads_started":1}
I20260926 20:41:47.431744 2974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:47.410497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:35617 (local address 127.23.44.194:44395)
0926 20:41:47.411456 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0926 20:41:47.411505 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 20:41:47.413257 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:41:47.414042 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:41:47.414050 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:41:47.414542 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:41:47.415332 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:47.415357 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:47.417159 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:47.417163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:47.418448 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:47.418468 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:47.419811 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:47.419816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:47.420450 (+ 634us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:47.421243 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:41:47.421277 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:41:47.423900 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:41:47.427173 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:41:47.427183 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:41:47.427202 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:41:47.427761 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:41:47.428148 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:41:47.428153 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:41:47.428163 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:41:47.428433 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:41:47.429394 (+ 961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:41:47.429404 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:41:47.429889 (+ 485us) client_negotiation.cc:769] Sending connection context
0926 20:41:47.430197 (+ 308us) client_negotiation.cc:240] Negotiation successful
0926 20:41:47.430609 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":393,"threads_started":1}
I20260926 20:41:47.433650 2971 heartbeater.cc:347] Connected to a master server at 127.23.44.254:35617
I20260926 20:41:47.433997 2971 heartbeater.cc:464] Registering TS with master...
I20260926 20:41:47.434902 2971 heartbeater.cc:511] Master 127.23.44.254:35617 requested a full tablet report, sending...
I20260926 20:41:47.437104 2615 ts_manager.cc:194] Registered new tserver with Master: e479e166f72c419fb82d306a08c10694 (127.23.44.194:41069)
I20260926 20:41:47.437906 2615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:44395
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:41:47.471741 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.44.212:36869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:35617
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:41:47.634894 2828 heartbeater.cc:503] Master 127.23.44.254:35617 was elected leader, sending a full tablet report...
W20260926 20:41:47.859504 2977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:41:47.859869 2977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:41:47.860034 2977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:41:47.868769 2977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:41:47.868929 2977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:41:47.869096 2977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:41:47.879915 2977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.44.212:36869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:35617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:41:47.881335 2977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:41:47.882630 2977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:41:47.896420 2982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:41:47.896350 2985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:41:47.896445 2983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:41:47.897578 2977 server_base.cc:1034] running on GCE node
I20260926 20:41:47.898303 2977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:41:47.899379 2977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:41:47.900665 2977 hybrid_clock.cc:648] HybridClock initialized: now 1790455307900597 us; error 61 us; skew 500 ppm
I20260926 20:41:47.900990 2977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:41:47 dist-test-slave-4d5t krb5kdc[2138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455307, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:41:47.904899 2977 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:41:47.906548 2977 webserver.cc:460] Webserver started at http://127.23.44.195:33255/ using document root <none> and password file <none>
I20260926 20:41:47.907266 2977 fs_manager.cc:359] Metadata directory not provided
I20260926 20:41:47.907430 2977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:41:47.907755 2977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:41:47 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455307, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:41:47.922778 2977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "3ae08f96214c4c12bc33152eb4c5e389"
format_stamp: "Formatted at 2026-09-26 20:41:47 on dist-test-slave-4d5t"
server_key: "74793f85f4aebb7914c1bc0d4bec1bb8"
server_key_iv: "c51d9411ae9de43a13b06f78055f5ce7"
server_key_version: "kuduclusterkey@0"
I20260926 20:41:47.923677 2977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "3ae08f96214c4c12bc33152eb4c5e389"
format_stamp: "Formatted at 2026-09-26 20:41:47 on dist-test-slave-4d5t"
server_key: "74793f85f4aebb7914c1bc0d4bec1bb8"
server_key_iv: "c51d9411ae9de43a13b06f78055f5ce7"
server_key_version: "kuduclusterkey@0"
I20260926 20:41:47.929894 2977 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260926 20:41:47.969313 2992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:41:47.971604 2977 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.010s sys 0.009s
I20260926 20:41:47.971838 2977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "3ae08f96214c4c12bc33152eb4c5e389"
format_stamp: "Formatted at 2026-09-26 20:41:47 on dist-test-slave-4d5t"
server_key: "74793f85f4aebb7914c1bc0d4bec1bb8"
server_key_iv: "c51d9411ae9de43a13b06f78055f5ce7"
server_key_version: "kuduclusterkey@0"
I20260926 20:41:47.972114 2977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:41:48.003547 2977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:41:48.008239 2977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:41:48.008620 2977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:41:48.009760 2977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:41:48.011333 2977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:41:48.011410 2977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:41:48.011564 2977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:41:48.011626 2977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:41:48.070109 2977 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:34505
I20260926 20:41:48.070122 3104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:34505 every 8 connection(s)
I20260926 20:41:48.071604 2977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:41:48.072371 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 2977
I20260926 20:41:48.072599 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:41:48.073272 23731 external_mini_cluster.cc:1428] Setting key 5e5315afde8491533eeb962761c63192
Sep 26 20:41:48 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455307, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:41:48.103952 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.074718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:54671 (local address 127.23.44.254:35617)
0926 20:41:48.081020 (+ 6302us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:48.081029 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:48.082442 (+ 1413us) server_negotiation.cc:407] Connection header received
0926 20:41:48.084788 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:48.084797 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:48.084920 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:48.085093 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:41:48.086725 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.088091 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.090431 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.090949 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.091100 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.095346 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:41:48.095356 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:41:48.095364 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:41:48.095441 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:41:48.100104 (+ 4663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:48.101119 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:48.101127 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:48.101133 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:48.101345 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:48.101806 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:48.101814 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:48.101819 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:48.102263 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:41:48.102566 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:48.103158 (+ 592us) server_negotiation.cc:299] Negotiation successful
0926 20:41:48.103458 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6152,"thread_start_us":6028,"threads_started":1}
I20260926 20:41:48.104703 3108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.075676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:35617 (local address 127.23.44.195:54671)
0926 20:41:48.081880 (+ 6204us) negotiation.cc:107] Waiting for socket to connect
0926 20:41:48.081932 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 20:41:48.084431 (+ 2499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:41:48.085285 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:41:48.085294 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:41:48.085753 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:41:48.086522 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:48.086542 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.088287 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.088292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:48.090212 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:48.090227 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.091201 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.091210 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:48.091489 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.092376 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:41:48.092399 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:41:48.095044 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:41:48.100341 (+ 5297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:41:48.100350 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:41:48.100367 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:41:48.100960 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:41:48.101473 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:41:48.101477 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:41:48.101483 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:41:48.101678 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:41:48.102390 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:41:48.102397 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:41:48.102742 (+ 345us) client_negotiation.cc:769] Sending connection context
0926 20:41:48.102985 (+ 243us) client_negotiation.cc:240] Negotiation successful
0926 20:41:48.103391 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5856,"thread_start_us":5578,"threads_started":1}
I20260926 20:41:48.106405 3105 heartbeater.cc:347] Connected to a master server at 127.23.44.254:35617
I20260926 20:41:48.106778 3105 heartbeater.cc:464] Registering TS with master...
I20260926 20:41:48.107769 3105 heartbeater.cc:511] Master 127.23.44.254:35617 requested a full tablet report, sending...
I20260926 20:41:48.109862 2615 ts_manager.cc:194] Registered new tserver with Master: 3ae08f96214c4c12bc33152eb4c5e389 (127.23.44.195:34505)
I20260926 20:41:48.110562 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:41:48.111428 2615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:54671
I20260926 20:41:48.135160 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.116005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54922 (local address 127.23.44.254:35617)
0926 20:41:48.116247 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:48.116255 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:48.117006 (+ 751us) server_negotiation.cc:407] Connection header received
0926 20:41:48.117227 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:48.117231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:48.117313 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:48.117406 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:41:48.120074 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.121130 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.123786 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.124480 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.124748 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.128527 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:41:48.128536 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:41:48.128541 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:41:48.128628 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:41:48.131445 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:48.132728 (+ 1283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:48.132732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:48.132734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:48.132889 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:48.133428 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:48.133432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:48.133435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:48.133773 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:41:48.133966 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:48.134453 (+ 487us) server_negotiation.cc:299] Negotiation successful
0926 20:41:48.134644 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:41:48.140718 2615 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:41:48.155920 2615 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:41:48.208731 3118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.172127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:36493 (local address 127.0.0.1:44786)
0926 20:41:48.185956 (+ 13829us) negotiation.cc:107] Waiting for socket to connect
0926 20:41:48.186023 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0926 20:41:48.186298 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:41:48.200753 (+ 14455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:41:48.200760 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:41:48.200788 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:41:48.201205 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:48.201216 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.203460 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.203466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:48.205680 (+ 2214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:48.205697 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.207632 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.207638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:48.207895 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.207952 (+ 57us) client_negotiation.cc:769] Sending connection context
0926 20:41:48.208055 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 20:41:48.208256 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13465,"mutex_wait_us":1,"thread_start_us":13241,"threads_started":1}
I20260926 20:41:48.210860 3119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.172861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44786 (local address 127.23.44.193:36493)
0926 20:41:48.199890 (+ 27029us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:48.199899 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:48.199932 (+ 33us) server_negotiation.cc:407] Connection header received
0926 20:41:48.200041 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:48.200053 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:48.200313 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:48.201437 (+ 1124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:41:48.201705 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.203300 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.205907 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.207398 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.209507 (+ 2109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.209760 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:48.209903 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 20:41:48.210479 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26828,"thread_start_us":26840,"threads_started":1}
I20260926 20:41:48.213771 3120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.173079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44078 (local address 127.23.44.195:34505)
0926 20:41:48.196834 (+ 23755us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:48.196850 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:48.196897 (+ 47us) server_negotiation.cc:407] Connection header received
0926 20:41:48.197034 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:48.197048 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:48.197441 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:48.197711 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:41:48.201018 (+ 3307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.202838 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.208438 (+ 5600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.210636 (+ 2198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.210746 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.210958 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:48.213096 (+ 2138us) server_negotiation.cc:299] Negotiation successful
0926 20:41:48.213365 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23480,"thread_start_us":27402,"threads_started":1}
I20260926 20:41:48.214881 3122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.173451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:34505 (local address 127.0.0.1:44078)
0926 20:41:48.194636 (+ 21185us) negotiation.cc:107] Waiting for socket to connect
0926 20:41:48.194667 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 20:41:48.194875 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:41:48.198179 (+ 3304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:41:48.198191 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:41:48.198248 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:41:48.199288 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:48.199303 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.203021 (+ 3718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.203027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:48.208234 (+ 5207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:48.208248 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.212627 (+ 4379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.212635 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:48.212828 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.212862 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 20:41:48.214282 (+ 1420us) client_negotiation.cc:240] Negotiation successful
0926 20:41:48.214455 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20986,"mutex_wait_us":2787,"thread_start_us":21541,"threads_started":1}
I20260926 20:41:48.215696 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.193257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:41069 (local address 127.0.0.1:38458)
0926 20:41:48.202533 (+ 9276us) negotiation.cc:107] Waiting for socket to connect
0926 20:41:48.202556 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:41:48.202701 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:41:48.206474 (+ 3773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:41:48.206482 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:41:48.206517 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:41:48.207033 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:48.207047 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.209384 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.209392 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:48.211922 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:48.211941 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.214614 (+ 2673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.214623 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:48.214858 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.214911 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 20:41:48.215010 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 20:41:48.215187 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9120,"thread_start_us":9515,"threads_started":1}
I20260926 20:41:48.215495 2763 tablet_service.cc:1467] Processing CreateTablet for tablet 6dd58b0fc4f04e42975a05f12a43f80c (DEFAULT_TABLE table=test-table [id=d8768a1b58434d00b695f922aee26bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:41:48.217013 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.186846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38458 (local address 127.23.44.194:41069)
0926 20:41:48.205722 (+ 18876us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:48.205732 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:48.205763 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:41:48.205901 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:48.205910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:48.206220 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:48.206468 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:41:48.207296 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.209190 (+ 1894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.212138 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.214018 (+ 1880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.214193 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.214430 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:48.216330 (+ 1900us) server_negotiation.cc:299] Negotiation successful
0926 20:41:48.216567 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18679,"thread_start_us":18559,"threads_started":1}
I20260926 20:41:48.217190 2763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dd58b0fc4f04e42975a05f12a43f80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:41:48.219038 3040 tablet_service.cc:1467] Processing CreateTablet for tablet 6dd58b0fc4f04e42975a05f12a43f80c (DEFAULT_TABLE table=test-table [id=d8768a1b58434d00b695f922aee26bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:41:48.221014 2906 tablet_service.cc:1467] Processing CreateTablet for tablet 6dd58b0fc4f04e42975a05f12a43f80c (DEFAULT_TABLE table=test-table [id=d8768a1b58434d00b695f922aee26bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:41:48.222702 2906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dd58b0fc4f04e42975a05f12a43f80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:41:48.223184 3040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dd58b0fc4f04e42975a05f12a43f80c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:41:48.248790 3126 tablet_bootstrap.cc:492] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694: Bootstrap starting.
I20260926 20:41:48.249266 3125 tablet_bootstrap.cc:492] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4: Bootstrap starting.
I20260926 20:41:48.253690 3125 tablet_bootstrap.cc:654] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4: Neither blocks nor log segments found. Creating new log.
I20260926 20:41:48.255271 3125 log.cc:824] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4: Log is configured to *not* fsync() on all Append() calls
I20260926 20:41:48.255422 3126 tablet_bootstrap.cc:654] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694: Neither blocks nor log segments found. Creating new log.
I20260926 20:41:48.256421 3127 tablet_bootstrap.cc:492] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389: Bootstrap starting.
I20260926 20:41:48.260054 3126 log.cc:824] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694: Log is configured to *not* fsync() on all Append() calls
I20260926 20:41:48.261205 3125 tablet_bootstrap.cc:492] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4: No bootstrap required, opened a new log
I20260926 20:41:48.261591 3127 tablet_bootstrap.cc:654] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389: Neither blocks nor log segments found. Creating new log.
I20260926 20:41:48.263495 3125 ts_tablet_manager.cc:1397] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.005s
I20260926 20:41:48.264050 3127 log.cc:824] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389: Log is configured to *not* fsync() on all Append() calls
I20260926 20:41:48.271101 3126 tablet_bootstrap.cc:492] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694: No bootstrap required, opened a new log
I20260926 20:41:48.271322 3127 tablet_bootstrap.cc:492] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389: No bootstrap required, opened a new log
I20260926 20:41:48.271505 3126 ts_tablet_manager.cc:1397] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694: Time spent bootstrapping tablet: real 0.023s user 0.012s sys 0.004s
I20260926 20:41:48.271638 3127 ts_tablet_manager.cc:1397] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260926 20:41:48.275614 3125 raft_consensus.cc:348] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36493 } } peers { permanent_uuid: "e479e166f72c419fb82d306a08c10694" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41069 } } peers { permanent_uuid: "3ae08f96214c4c12bc33152eb4c5e389" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 34505 } }
I20260926 20:41:48.276140 3125 raft_consensus.cc:374] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:41:48.276283 3125 raft_consensus.cc:729] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70d89cb65c74999aaa3cf57c27b62d4, State: Initialized, Role: FOLLOWER
I20260926 20:41:48.277165 3125 consensus_queue.cc:260] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36493 } } peers { permanent_uuid: "e479e166f72c419fb82d306a08c10694" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41069 } } peers { permanent_uuid: "3ae08f96214c4c12bc33152eb4c5e389" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 34505 } }
I20260926 20:41:48.281244 3125 ts_tablet_manager.cc:1428] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4: Time spent starting tablet: real 0.018s user 0.014s sys 0.003s
I20260926 20:41:48.281535 3126 raft_consensus.cc:348] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36493 } } peers { permanent_uuid: "e479e166f72c419fb82d306a08c10694" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41069 } } peers { permanent_uuid: "3ae08f96214c4c12bc33152eb4c5e389" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 34505 } }
I20260926 20:41:48.282022 3126 raft_consensus.cc:374] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:41:48.282228 3126 raft_consensus.cc:729] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e479e166f72c419fb82d306a08c10694, State: Initialized, Role: FOLLOWER
I20260926 20:41:48.282753 3127 raft_consensus.cc:348] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36493 } } peers { permanent_uuid: "e479e166f72c419fb82d306a08c10694" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41069 } } peers { permanent_uuid: "3ae08f96214c4c12bc33152eb4c5e389" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 34505 } }
I20260926 20:41:48.282873 3126 consensus_queue.cc:260] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36493 } } peers { permanent_uuid: "e479e166f72c419fb82d306a08c10694" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41069 } } peers { permanent_uuid: "3ae08f96214c4c12bc33152eb4c5e389" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 34505 } }
I20260926 20:41:48.283097 3127 raft_consensus.cc:374] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:41:48.283245 3127 raft_consensus.cc:729] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae08f96214c4c12bc33152eb4c5e389, State: Initialized, Role: FOLLOWER
I20260926 20:41:48.283886 3127 consensus_queue.cc:260] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36493 } } peers { permanent_uuid: "e479e166f72c419fb82d306a08c10694" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41069 } } peers { permanent_uuid: "3ae08f96214c4c12bc33152eb4c5e389" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 34505 } }
I20260926 20:41:48.285531 2971 heartbeater.cc:503] Master 127.23.44.254:35617 was elected leader, sending a full tablet report...
I20260926 20:41:48.285657 3127 ts_tablet_manager.cc:1428] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260926 20:41:48.286026 3105 heartbeater.cc:503] Master 127.23.44.254:35617 was elected leader, sending a full tablet report...
I20260926 20:41:48.286115 3126 ts_tablet_manager.cc:1428] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
W20260926 20:41:48.329907 3106 tablet.cc:2369] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:41:48.342303 2829 tablet.cc:2369] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:41:48.364826 3131 raft_consensus.cc:482] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:41:48.365167 3131 raft_consensus.cc:504] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36493 } } peers { permanent_uuid: "e479e166f72c419fb82d306a08c10694" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41069 } } peers { permanent_uuid: "3ae08f96214c4c12bc33152eb4c5e389" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 34505 } }
I20260926 20:41:48.367159 3131 leader_election.cc:288] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e479e166f72c419fb82d306a08c10694 (127.23.44.194:41069), 3ae08f96214c4c12bc33152eb4c5e389 (127.23.44.195:34505)
I20260926 20:41:48.377909 3120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.367848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:55721 (local address 127.23.44.195:34505)
0926 20:41:48.368085 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:48.368093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:48.369243 (+ 1150us) server_negotiation.cc:407] Connection header received
0926 20:41:48.369508 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:48.369517 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:48.369607 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:48.369715 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:41:48.370385 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.371735 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.375081 (+ 3346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.376686 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.376876 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.376957 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:48.377500 (+ 543us) server_negotiation.cc:299] Negotiation successful
0926 20:41:48.377642 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260926 20:41:48.377909 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.367665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:47885 (local address 127.23.44.194:41069)
0926 20:41:48.367891 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:48.367899 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:48.369562 (+ 1663us) server_negotiation.cc:407] Connection header received
0926 20:41:48.369713 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:48.369719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:48.369799 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:48.369899 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:41:48.370547 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.371772 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.375084 (+ 3312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.376686 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.376953 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.377032 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:48.377499 (+ 467us) server_negotiation.cc:299] Negotiation successful
0926 20:41:48.377642 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:41:48.378137 3135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.368014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:41069 (local address 127.23.44.193:47885)
0926 20:41:48.369450 (+ 1436us) negotiation.cc:107] Waiting for socket to connect
0926 20:41:48.369465 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 20:41:48.369581 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:41:48.369939 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:41:48.369949 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:41:48.369969 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:41:48.370398 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:48.370409 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.371943 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.371949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:48.374766 (+ 2817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:48.374786 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.377025 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.377031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:48.377270 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.377351 (+ 81us) client_negotiation.cc:769] Sending connection context
0926 20:41:48.377456 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 20:41:48.377642 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1304,"mutex_wait_us":322,"thread_start_us":1146,"threads_started":1}
I20260926 20:41:48.378162 3134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.367541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:34505 (local address 127.23.44.193:55721)
0926 20:41:48.369105 (+ 1564us) negotiation.cc:107] Waiting for socket to connect
0926 20:41:48.369128 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:41:48.369353 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:41:48.369736 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:41:48.369740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:41:48.369757 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:41:48.370224 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:48.370239 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.371942 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.371949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:48.374771 (+ 2822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:41:48.374786 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.377007 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.377015 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:41:48.377269 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.377351 (+ 82us) client_negotiation.cc:769] Sending connection context
0926 20:41:48.377456 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 20:41:48.377642 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1386,"mutex_wait_us":239,"thread_start_us":1152,"threads_started":1}
I20260926 20:41:48.379411 3060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dd58b0fc4f04e42975a05f12a43f80c" candidate_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae08f96214c4c12bc33152eb4c5e389" is_pre_election: true
I20260926 20:41:48.379922 3060 raft_consensus.cc:2454] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e70d89cb65c74999aaa3cf57c27b62d4 in term 0.
I20260926 20:41:48.379851 2926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dd58b0fc4f04e42975a05f12a43f80c" candidate_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e479e166f72c419fb82d306a08c10694" is_pre_election: true
I20260926 20:41:48.380357 2926 raft_consensus.cc:2454] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e70d89cb65c74999aaa3cf57c27b62d4 in term 0.
I20260926 20:41:48.380656 2719 leader_election.cc:302] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae08f96214c4c12bc33152eb4c5e389, e70d89cb65c74999aaa3cf57c27b62d4; no voters:
I20260926 20:41:48.381132 3131 raft_consensus.cc:2781] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:41:48.381284 3131 raft_consensus.cc:482] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:41:48.381412 3131 raft_consensus.cc:3037] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:41:48.383625 3131 raft_consensus.cc:504] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36493 } } peers { permanent_uuid: "e479e166f72c419fb82d306a08c10694" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41069 } } peers { permanent_uuid: "3ae08f96214c4c12bc33152eb4c5e389" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 34505 } }
I20260926 20:41:48.384238 3131 leader_election.cc:288] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [CANDIDATE]: Term 1 election: Requested vote from peers e479e166f72c419fb82d306a08c10694 (127.23.44.194:41069), 3ae08f96214c4c12bc33152eb4c5e389 (127.23.44.195:34505)
I20260926 20:41:48.384747 2926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dd58b0fc4f04e42975a05f12a43f80c" candidate_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e479e166f72c419fb82d306a08c10694"
I20260926 20:41:48.384744 3060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dd58b0fc4f04e42975a05f12a43f80c" candidate_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae08f96214c4c12bc33152eb4c5e389"
I20260926 20:41:48.384956 2926 raft_consensus.cc:3037] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:41:48.384964 3060 raft_consensus.cc:3037] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:41:48.387064 3060 raft_consensus.cc:2454] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e70d89cb65c74999aaa3cf57c27b62d4 in term 1.
I20260926 20:41:48.387347 2926 raft_consensus.cc:2454] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e70d89cb65c74999aaa3cf57c27b62d4 in term 1.
I20260926 20:41:48.387514 2719 leader_election.cc:302] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ae08f96214c4c12bc33152eb4c5e389, e70d89cb65c74999aaa3cf57c27b62d4; no voters:
I20260926 20:41:48.387847 3131 raft_consensus.cc:2781] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:41:48.388242 3131 raft_consensus.cc:686] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [term 1 LEADER]: Becoming Leader. State: Replica: e70d89cb65c74999aaa3cf57c27b62d4, State: Running, Role: LEADER
I20260926 20:41:48.388855 3131 consensus_queue.cc:237] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36493 } } peers { permanent_uuid: "e479e166f72c419fb82d306a08c10694" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41069 } } peers { permanent_uuid: "3ae08f96214c4c12bc33152eb4c5e389" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 34505 } }
I20260926 20:41:48.394127 2613 catalog_manager.cc:5495] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e70d89cb65c74999aaa3cf57c27b62d4 (127.23.44.193). New cstate: current_term: 1 leader_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70d89cb65c74999aaa3cf57c27b62d4" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 36493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e479e166f72c419fb82d306a08c10694" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae08f96214c4c12bc33152eb4c5e389" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 34505 } health_report { overall_health: UNKNOWN } } }
W20260926 20:41:48.410094 2972 tablet.cc:2369] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:41:48.451709 3119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.446086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.23.44.193:36493)
0926 20:41:48.446334 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:48.446339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:48.446360 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:41:48.446538 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:48.446542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:48.446631 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:48.446770 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:41:48.447144 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.447899 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.449343 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.449738 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.449846 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.450509 (+ 663us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:41:48.451098 (+ 589us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:41:48.451233 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:48.451343 (+ 110us) server_negotiation.cc:299] Negotiation successful
0926 20:41:48.451411 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 20:41:48.461485 2926 raft_consensus.cc:1261] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694 [term 1 FOLLOWER]: Refusing update from remote peer e70d89cb65c74999aaa3cf57c27b62d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:41:48.461783 3060 raft_consensus.cc:1261] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389 [term 1 FOLLOWER]: Refusing update from remote peer e70d89cb65c74999aaa3cf57c27b62d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:41:48.463486 3136 consensus_queue.cc:1035] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e479e166f72c419fb82d306a08c10694" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:41:48.464113 3131 consensus_queue.cc:1035] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae08f96214c4c12bc33152eb4c5e389" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 34505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:41:48.501250 2613 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:41:48.501614 2613 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:41:48.506155 2613 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe912c4c103a48c0a37e65f46c602f2f: Sending DeleteTablet for 3 replicas of tablet 6dd58b0fc4f04e42975a05f12a43f80c
I20260926 20:41:48.507957 2763 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dd58b0fc4f04e42975a05f12a43f80c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:41:48 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:44786
I20260926 20:41:48.508133 3040 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dd58b0fc4f04e42975a05f12a43f80c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:41:48 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:44078
I20260926 20:41:48.508370 2906 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dd58b0fc4f04e42975a05f12a43f80c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:41:48 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:38458
I20260926 20:41:48.512626 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 2695
I20260926 20:41:48.518489 3151 tablet_replica.cc:330] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694: stopping tablet replica
I20260926 20:41:48.519347 3151 raft_consensus.cc:2229] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:41:48.520159 3151 raft_consensus.cc:2258] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:41:48.521273 3150 tablet_replica.cc:330] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389: stopping tablet replica
I20260926 20:41:48.524209 3151 ts_tablet_manager.cc:1905] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:41:48.524858 3150 raft_consensus.cc:2229] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:41:48.525394 3150 raft_consensus.cc:2258] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:41:48.527689 3150 ts_tablet_manager.cc:1905] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:41:48.528530 3149 tablet_replica.cc:330] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4: stopping tablet replica
I20260926 20:41:48.529171 3149 raft_consensus.cc:2229] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:41:48.529979 3149 raft_consensus.cc:2258] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:41:48.536052 3150 ts_tablet_manager.cc:1918] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:41:48.536124 3151 ts_tablet_manager.cc:1918] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:41:48.536311 3151 log.cc:1196] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/6dd58b0fc4f04e42975a05f12a43f80c
I20260926 20:41:48.536787 3150 log.cc:1196] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/6dd58b0fc4f04e42975a05f12a43f80c
I20260926 20:41:48.537505 3150 ts_tablet_manager.cc:1939] T 6dd58b0fc4f04e42975a05f12a43f80c P 3ae08f96214c4c12bc33152eb4c5e389: Deleting consensus metadata
I20260926 20:41:48.539152 3149 ts_tablet_manager.cc:1905] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 20:41:48 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:41:48.540222 2602 catalog_manager.cc:4841] TS 3ae08f96214c4c12bc33152eb4c5e389 (127.23.44.195:34505): tablet 6dd58b0fc4f04e42975a05f12a43f80c (table test-table [id=d8768a1b58434d00b695f922aee26bb0]) successfully deleted
I20260926 20:41:48.541657 3151 ts_tablet_manager.cc:1939] T 6dd58b0fc4f04e42975a05f12a43f80c P e479e166f72c419fb82d306a08c10694: Deleting consensus metadata
I20260926 20:41:48.543900 2599 catalog_manager.cc:4841] TS e479e166f72c419fb82d306a08c10694 (127.23.44.194:41069): tablet 6dd58b0fc4f04e42975a05f12a43f80c (table test-table [id=d8768a1b58434d00b695f922aee26bb0]) successfully deleted
I20260926 20:41:48.549450 3119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.514249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44802 (local address 127.23.44.193:36493)
0926 20:41:48.514529 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:48.514539 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:48.524516 (+ 9977us) server_negotiation.cc:407] Connection header received
0926 20:41:48.524813 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:48.524822 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:48.524941 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:48.525080 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:41:48.530628 (+ 5548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.532063 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.534242 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.534933 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.535055 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.541446 (+ 6391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:41:48.541466 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:41:48.541488 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:41:48.541554 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:41:48.546057 (+ 4503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:48.546971 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:48.546979 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:48.546982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:48.547162 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:48.547606 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:48.547611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:48.547613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:48.548000 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:41:48.548425 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:48.548747 (+ 322us) server_negotiation.cc:299] Negotiation successful
0926 20:41:48.549022 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260926 20:41:48.549692 3149 ts_tablet_manager.cc:1918] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:41:48.549887 3149 log.cc:1196] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/6dd58b0fc4f04e42975a05f12a43f80c
I20260926 20:41:48.550367 2823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:41:48.657166 3149 ts_tablet_manager.cc:1939] T 6dd58b0fc4f04e42975a05f12a43f80c P e70d89cb65c74999aaa3cf57c27b62d4: Deleting consensus metadata
I20260926 20:41:48.659391 2599 catalog_manager.cc:4841] TS e70d89cb65c74999aaa3cf57c27b62d4 (127.23.44.193:36493): tablet 6dd58b0fc4f04e42975a05f12a43f80c (table test-table [id=d8768a1b58434d00b695f922aee26bb0]) successfully deleted
I20260926 20:41:48.664110 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 2695
I20260926 20:41:48.685887 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 2834
Sep 26 20:41:48 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:41:48.705422 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.686583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38468 (local address 127.23.44.194:41069)
0926 20:41:48.686755 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:48.686759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:48.686833 (+ 74us) server_negotiation.cc:407] Connection header received
0926 20:41:48.687044 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:48.687049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:48.687124 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:48.687210 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:41:48.688475 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.689325 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.690646 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.691037 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.691167 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.698990 (+ 7823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:41:48.699009 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:41:48.699026 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:41:48.699066 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:41:48.702408 (+ 3342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:48.703229 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:48.703236 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:48.703239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:48.703396 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:48.703780 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:48.703784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:48.703787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:48.704246 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:41:48.704621 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:48.704865 (+ 244us) server_negotiation.cc:299] Negotiation successful
0926 20:41:48.705066 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 20:41:48.706137 2966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:41:48.802561 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 2834
I20260926 20:41:48.820259 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 2977
Sep 26 20:41:48 dist-test-slave-4d5t krb5kdc[2138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:41:48.836671 3120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:41:48.821419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44094 (local address 127.23.44.195:34505)
0926 20:41:48.821599 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 20:41:48.821607 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:41:48.821630 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:41:48.821877 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:41:48.821885 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:41:48.821973 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:41:48.822090 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:41:48.823527 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.824365 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.825715 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:41:48.826105 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:41:48.826247 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:41:48.830813 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:41:48.830828 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:41:48.830844 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:41:48.830914 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:41:48.833744 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:48.834486 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:48.834503 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:48.834506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:48.834638 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:41:48.835029 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:41:48.835035 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:41:48.835038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:41:48.835424 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:41:48.835734 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0926 20:41:48.836031 (+ 297us) server_negotiation.cc:299] Negotiation successful
0926 20:41:48.836247 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 20:41:48.837320 3100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:41:48.928973 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 2977
I20260926 20:41:48.945874 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 2583
I20260926 20:41:48.947083 2645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:41:49.072340 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 2583
I20260926 20:41:49.099033 23731 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 20:41:49.445858 23731 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 20:41:49.446118 23731 mini_ranger.cc:66] Stopping Ranger...
I20260926 20:41:49.792699 23731 mini_ranger.cc:68] Stopped Ranger
2026-09-26 20:41:49.793 UTC [2168] LOG: received smart shutdown request
2026-09-26 20:41:49.797 UTC [2168] LOG: background worker "logical replication launcher" (PID 2177) exited with exit code 1
2026-09-26 20:41:49.797 UTC [2172] LOG: shutting down
2026-09-26 20:41:49.978 UTC [2168] LOG: database system is shut down
2026-09-26T20:41:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53833 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:41:50 dist-test-slave-4d5t krb5kdc[3165](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:41:50 dist-test-slave-4d5t krb5kdc[3165](info): set up 2 sockets
Sep 26 20:41:50 dist-test-slave-4d5t krb5kdc[3165](info): commencing operation
krb5kdc: starting...
W20260926 20:41:51.274101 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.007s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:41:51 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:41:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:41:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/rangeradmin_127.23.44.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/rangeradmin_127.23.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/rangerlookup_127.23.44.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/rangerlookup_127.23.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/HTTP_127.23.44.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/HTTP_127.23.44.212@KRBTEST.COM.keytab.
I20260926 20:41:51.635217 23731 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskO4QBsv/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/postgres -l logfile start
2026-09-26 20:41:54.348 UTC [3195] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 20:41:54.349 UTC [3195] LOG: listening on IPv4 address "127.23.44.212", port 41957
2026-09-26 20:41:54.352 UTC [3195] LOG: listening on Unix socket "/tmp/.s.PGSQL.41957"
2026-09-26 20:41:54.373 UTC [3198] LOG: database system was shut down at 2026-09-26 20:41:52 UTC
2026-09-26 20:41:54.377 UTC [3195] LOG: database system is ready to accept connections
I20260926 20:41:55.405299 23731 mini_postgres.cc:96] Postgres bound to 41957
2026-09-26 20:41:55.422 UTC [3206] FATAL: database "slave" does not exist
127.23.44.212:41957 - accepting connections
I20260926 20:41:55.423655 23731 mini_ranger.cc:161] Starting Ranger...
I20260926 20:41:55.453894 23731 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 20:41:56.012506 23731 mini_ranger.cc:87] Created ranger Postgres database
I20260926 20:41:56.012749 23731 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-admin
2026-09-26 20:41:56,319 [I] DB FLAVOR :POSTGRES
2026-09-26 20:41:56,320 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 20:41:56,320 [I] Checking connection..
2026-09-26 20:41:56,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 20:41:56,625 [I] Checking connection passed.
2026-09-26 20:41:56,626 [I] --------- Verifying version history table ---------
2026-09-26 20:41:56,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 20:41:56,902 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 20:41:56,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 20:41:57,169 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 20:41:57,170 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 20:41:57,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 20:41:57.437 UTC [3270] WARNING: there is no transaction in progress
2026-09-26 20:41:57,444 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 20:41:57,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 20:41:57,717 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 20:41:57,717 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 20:41:57,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 20:41:57,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 20:41:58,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4d5t.c.gcp-upstream.internal','N') ;"
2026-09-26 20:41:58,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 20:41:58,804 [I] Table x_portal_user does not exist in database ranger
2026-09-26 20:41:58,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 20:41:59,087 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 20:41:59,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 20:41:59,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 20:41:59,639 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 20:41:59,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 20:42:00.746 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:00.767 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:00.788 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:00.807 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:00.832 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:00.852 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:00.919 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:00.928 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:00.945 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:00.957 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:00.970 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:00.987 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:01.000 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:01.008 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:01.016 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:01.024 UTC [3378] WARNING: there is no transaction in progress
2026-09-26 20:42:01,817 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 20:42:01,818 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 20:42:02,081 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 20:42:02,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 20:42:02,343 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 20:42:02,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 20:42:02,604 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 20:42:02,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 20:42:02,860 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 20:42:03,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4d5t.c.gcp-upstream.internal';"
2026-09-26 20:42:03,389 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 20:42:03,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 20:42:03,654 [I] DB_PATCHES have already been applied
I20260926 20:42:03.661487 23731 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-admin:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskO4QBsv/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 20:42:03.661598 23731 mini_ranger.cc:193] Using host: 127.23.44.212
I20260926 20:42:03.677289 23731 mini_ranger.cc:227] Ranger admin URL: http://127.23.44.212:45595
Sep 26, 2026 8:42:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 8:42:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 8:42:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 8:42:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 8:42:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 8:42:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 8:42:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 20:42:04.872190 23731 mini_ranger.cc:160] Time spent starting Ranger: real 9.449s user 0.001s sys 0.044s
Sep 26, 2026 8:42:27 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 8:42:27 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 8:42:27 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 8:42:27 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 8:42:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 8:42:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 8:42:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 8:42:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 8:42:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 8:42:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 8:42:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 8:42:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 8:42:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 8:42:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 8:42:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 8:42:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 8:42:27 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 8:42:28 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 20:42:28.848224 23731 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.44.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.44.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/rangerkms_127.23.44.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.44.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/rangerkms_127.23.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.44.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.44.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.44.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/HTTP_127.23.44.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.44.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/HTTP_127.23.44.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 20:42:29 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455349, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 20:42:29.048107 23731 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 20:42:29.080268 23731 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 20:42:30.026439 23731 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 20:42:30.026539 23731 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-kms
I20260926 20:42:30.026573 23731 mini_ranger_kms.cc:227] Using postgres at 127.23.44.212:41957
2026-09-26 20:42:30,414 [I] DB FLAVOR :POSTGRES
2026-09-26 20:42:30,414 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 20:42:30,414 [I] Checking connection
2026-09-26 20:42:30,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 20:42:30,718 [I] connection success
2026-09-26 20:42:30,718 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 20:42:30,718 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 20:42:30,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 20:42:30,984 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 20:42:30,985 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 20:42:30,985 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 20:42:30,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.44.212:41957/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 20:42:31,287 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 20:42:31.831637 23731 mini_ranger_kms.cc:310] Created kms service
I20260926 20:42:32.026499 23731 mini_ranger_kms.cc:326] Created kudu user
I20260926 20:42:32.081602 23731 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 20:42:32 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455349, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:42:32.563722 23731 mini_ranger_kms.cc:384] Added ranger policy
I20260926 20:42:32.564298 23731 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-kms:/tmp/dist-test-taskO4QBsv/build/asan/bin/postgresql.jar:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskO4QBsv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 20:42:32.564358 23731 mini_ranger_kms.cc:240] Using host: 127.23.44.212
I20260926 20:42:32.583706 23731 mini_ranger_kms.cc:276] Ranger KMS PID: 3559
I20260926 20:42:32.583879 23731 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.44.212:35893
Sep 26, 2026 8:42:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 8:42:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 8:42:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 8:42:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 8:42:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 8:42:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 26 20:42:34 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790455354, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.44.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:34.999554 23731 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.951s user 0.074s sys 0.328s
I20260926 20:42:34.999794 23731 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 20:42:34.999948 23731 mini_ranger_kms.cc:401] 127.23.44.212:35893/kms/v1/keys
Sep 26 20:42:35 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455354, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.44.212@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
Sep 26 20:42:35 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455354, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.44.212@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
Sep 26 20:42:38 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:42:38.200109 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:34643
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:34643
--ranger_config_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.44.212:35893/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:42:38.544520 3609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:38.544893 3609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:38.544987 3609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:38.552511 3609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:42:38.552706 3609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:38.552757 3609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:38.552842 3609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:42:38.552927 3609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:42:38.563679 3609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.44.212:35893/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:34643
--ranger_config_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:34643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:38.565088 3609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:38.566424 3609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:38.579424 3615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:38.579416 3617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:38.579380 3614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:38.580211 3609 server_base.cc:1034] running on GCE node
I20260926 20:42:38.581199 3609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:38.582867 3609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:38.584146 3609 hybrid_clock.cc:648] HybridClock initialized: now 1790455358584063 us; error 86 us; skew 500 ppm
I20260926 20:42:38.584635 3609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:38 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455358, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:38.592044 3609 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:42:38.594127 3609 webserver.cc:460] Webserver started at http://127.23.44.254:43573/ using document root <none> and password file <none>
I20260926 20:42:38.594921 3609 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:38.595084 3609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:42:38.595443 3609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:42:38 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455358, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:42:38.647648 3609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "c942f3fa1b3949388f7a6569e3206161"
format_stamp: "Formatted at 2026-09-26 20:42:38 on dist-test-slave-4d5t"
server_key: "1811b732329af39f4008adfa6bab2cd4"
server_key_iv: "bf8ec87b7e3ef22289d7f9f55f9e4829"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:38.649690 3609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "c942f3fa1b3949388f7a6569e3206161"
format_stamp: "Formatted at 2026-09-26 20:42:38 on dist-test-slave-4d5t"
server_key: "1811b732329af39f4008adfa6bab2cd4"
server_key_iv: "bf8ec87b7e3ef22289d7f9f55f9e4829"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:38.657401 3609 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260926 20:42:38.679157 3624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:38.680830 3609 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:42:38.681033 3609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "c942f3fa1b3949388f7a6569e3206161"
format_stamp: "Formatted at 2026-09-26 20:42:38 on dist-test-slave-4d5t"
server_key: "1811b732329af39f4008adfa6bab2cd4"
server_key_iv: "bf8ec87b7e3ef22289d7f9f55f9e4829"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:38.681282 3609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:38.706828 3609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:38.712117 3609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:38.712492 3609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:38.748864 3609 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:34643
I20260926 20:42:38.748880 3675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:34643 every 8 connection(s)
I20260926 20:42:38.750557 3609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:42:38.755641 3676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:42:38.759028 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 3609
I20260926 20:42:38.759284 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:42:38.760339 23731 external_mini_cluster.cc:1428] Setting key 323b9d1818b0d9b56a2287d0418106fe
I20260926 20:42:38.771762 3676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161: Bootstrap starting.
I20260926 20:42:38.775506 3676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161: Neither blocks nor log segments found. Creating new log.
I20260926 20:42:38.776612 3676 log.cc:824] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161: Log is configured to *not* fsync() on all Append() calls
Sep 26 20:42:38 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:38.780146 3676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161: No bootstrap required, opened a new log
I20260926 20:42:38.787209 3679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:38.763331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46100 (local address 127.23.44.254:34643)
0926 20:42:38.764412 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:38.764429 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:38.764835 (+ 406us) server_negotiation.cc:407] Connection header received
0926 20:42:38.767147 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:38.767181 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:38.767753 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:38.768246 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:38.769616 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:38.771557 (+ 1941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:38.773418 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:38.774473 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:38.774686 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:38.779206 (+ 4520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:38.779214 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:38.779231 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:38.779268 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:38.782452 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:38.783457 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:38.783467 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:38.783475 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:38.783626 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:38.784052 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:38.784063 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:38.784065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:38.784759 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:38.785050 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:38.785575 (+ 525us) server_negotiation.cc:299] Negotiation successful
0926 20:42:38.786014 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":504,"threads_started":1}
I20260926 20:42:38.791013 3676 raft_consensus.cc:348] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c942f3fa1b3949388f7a6569e3206161" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34643 } }
I20260926 20:42:38.791404 3676 raft_consensus.cc:374] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:42:38.791512 3676 raft_consensus.cc:729] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c942f3fa1b3949388f7a6569e3206161, State: Initialized, Role: FOLLOWER
I20260926 20:42:38.792153 3676 consensus_queue.cc:260] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c942f3fa1b3949388f7a6569e3206161" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34643 } }
I20260926 20:42:38.792377 3676 raft_consensus.cc:388] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:42:38.792773 3676 raft_consensus.cc:482] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:42:38.793108 3676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:42:38.795862 3676 raft_consensus.cc:504] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c942f3fa1b3949388f7a6569e3206161" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34643 } }
I20260926 20:42:38.796460 3676 leader_election.cc:302] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c942f3fa1b3949388f7a6569e3206161; no voters:
I20260926 20:42:38.797303 3676 leader_election.cc:288] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:42:38.797477 3681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:42:38.798308 3681 raft_consensus.cc:686] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 1 LEADER]: Becoming Leader. State: Replica: c942f3fa1b3949388f7a6569e3206161, State: Running, Role: LEADER
I20260926 20:42:38.798990 3681 consensus_queue.cc:237] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c942f3fa1b3949388f7a6569e3206161" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34643 } }
I20260926 20:42:38.799628 3676 sys_catalog.cc:564] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:42:38.802378 3683 sys_catalog.cc:455] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c942f3fa1b3949388f7a6569e3206161. Latest consensus state: current_term: 1 leader_uuid: "c942f3fa1b3949388f7a6569e3206161" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c942f3fa1b3949388f7a6569e3206161" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34643 } } }
I20260926 20:42:38.802348 3682 sys_catalog.cc:455] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c942f3fa1b3949388f7a6569e3206161" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c942f3fa1b3949388f7a6569e3206161" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34643 } } }
I20260926 20:42:38.802883 3683 sys_catalog.cc:458] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [sys.catalog]: This master's current role is: LEADER
I20260926 20:42:38.802917 3682 sys_catalog.cc:458] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [sys.catalog]: This master's current role is: LEADER
I20260926 20:42:38.805188 3687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:42:38.810406 3687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:42:38.811975 3676 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 20:42:38.819787 3687 catalog_manager.cc:1347] Generated new cluster ID: 8f6e23b04a3b4edc99fb9bfbbffbfa4c
I20260926 20:42:38.819927 3687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:42:38.857251 3687 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:42:38.859875 3687 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:42:38.887295 3687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161: Generated new TSK 0
I20260926 20:42:38.888689 3687 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:42:39.667809 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.44.212:35893/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:34643
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:42:39 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790455359, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 20:42:40.085119 3721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:40.085588 3721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:40.085757 3721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:40.093549 3721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:40.093703 3721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:40.093884 3721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:42:40.104935 3721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.44.212:35893/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:34643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:40.106405 3721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:40.107808 3721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:40.129351 3731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:40.130702 3732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:40.137132 3735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:40.138585 3721 server_base.cc:1034] running on GCE node
I20260926 20:42:40.139451 3721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:40.141012 3721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:40.142402 3721 hybrid_clock.cc:648] HybridClock initialized: now 1790455360142327 us; error 74 us; skew 500 ppm
I20260926 20:42:40.142992 3721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:40 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455360, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:40.148649 3721 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:42:40.157697 3721 webserver.cc:460] Webserver started at http://127.23.44.193:41149/ using document root <none> and password file <none>
I20260926 20:42:40.158797 3721 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:40.158969 3721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:42:40.159349 3721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:42:40 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455360, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:42:40.180794 3721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "4d435588d72d4b3caa9a473404d634d6"
format_stamp: "Formatted at 2026-09-26 20:42:40 on dist-test-slave-4d5t"
server_key: "f2618d75c4f8d1a9dafe4b9e5c5b0809"
server_key_iv: "a735e6a30d57462d3bed38a1a72ee3e4"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:40.183889 3721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "4d435588d72d4b3caa9a473404d634d6"
format_stamp: "Formatted at 2026-09-26 20:42:40 on dist-test-slave-4d5t"
server_key: "f2618d75c4f8d1a9dafe4b9e5c5b0809"
server_key_iv: "a735e6a30d57462d3bed38a1a72ee3e4"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:40.193766 3721 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260926 20:42:40.236307 3741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:40.238644 3721 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.005s sys 0.009s
I20260926 20:42:40.238991 3721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "4d435588d72d4b3caa9a473404d634d6"
format_stamp: "Formatted at 2026-09-26 20:42:40 on dist-test-slave-4d5t"
server_key: "f2618d75c4f8d1a9dafe4b9e5c5b0809"
server_key_iv: "a735e6a30d57462d3bed38a1a72ee3e4"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:40.239248 3721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:40.280730 3721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:40.285446 3721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:40.285833 3721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:40.289103 3721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:42:40.290966 3721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:42:40.291079 3721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:40.291225 3721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:42:40.291311 3721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 20:42:40 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455359, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:42:40.814389 3721 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:41347
I20260926 20:42:40.814422 3853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:41347 every 8 connection(s)
I20260926 20:42:40.816032 3721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:42:40.824616 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 3721
I20260926 20:42:40.824887 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:42:40.825773 23731 external_mini_cluster.cc:1428] Setting key d84ba75feed2fb83f0d461b476712223
Sep 26 20:42:40 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455360, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
I20260926 20:42:40.861150 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:40.820488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:34643 (local address 127.23.44.193:42541)
0926 20:42:40.824906 (+ 4418us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:40.824958 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:40.827294 (+ 2336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:40.828250 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:40.828263 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:40.829042 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:40.829968 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:40.829996 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:40.831697 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:40.831703 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:40.833929 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:40.833947 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:40.835372 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:40.835380 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:40.835705 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:40.836660 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:40.836694 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:40.845839 (+ 9145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:40.855296 (+ 9457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:40.855309 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:40.855333 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:40.856124 (+ 791us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:40.856776 (+ 652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:40.856782 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:40.856786 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:40.857100 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:40.858019 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:40.858028 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:40.858548 (+ 520us) client_negotiation.cc:769] Sending connection context
0926 20:42:40.858895 (+ 347us) client_negotiation.cc:240] Negotiation successful
0926 20:42:40.859532 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4078,"thread_start_us":3743,"threads_started":1}
I20260926 20:42:40.862726 3856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:40.819568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:42541 (local address 127.23.44.254:34643)
0926 20:42:40.825807 (+ 6239us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:40.825815 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:40.825842 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:42:40.827579 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:40.827585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:40.827672 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:40.827780 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:40.830212 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:40.831454 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:40.834156 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:40.834892 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:40.835030 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:40.847511 (+ 12481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:40.847522 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:40.847528 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:40.847585 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:40.855051 (+ 7466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:40.856369 (+ 1318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:40.856377 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:40.856383 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:40.856607 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:40.857340 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:40.857346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:40.857350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:40.857754 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:40.857922 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:40.862010 (+ 4088us) server_negotiation.cc:299] Negotiation successful
0926 20:42:40.862294 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6087,"thread_start_us":7366,"threads_started":1}
I20260926 20:42:40.864331 3854 heartbeater.cc:347] Connected to a master server at 127.23.44.254:34643
I20260926 20:42:40.864735 3854 heartbeater.cc:464] Registering TS with master...
I20260926 20:42:40.865648 3854 heartbeater.cc:511] Master 127.23.44.254:34643 requested a full tablet report, sending...
I20260926 20:42:40.868119 3641 ts_manager.cc:194] Registered new tserver with Master: 4d435588d72d4b3caa9a473404d634d6 (127.23.44.193:41347)
I20260926 20:42:40.870307 3641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:42541
Sep 26 20:42:40 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455359, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:42:40.914422 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.44.212:35893/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:34643
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:42:41.347534 3861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:41.347886 3861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:41.348049 3861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:41.355665 3861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:41.355842 3861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:41.356016 3861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:42:41.367938 3861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.44.212:35893/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:34643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:41.369495 3861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:41.370934 3861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:41.386760 3874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:41.393993 3875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:41.394925 3877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:41.395032 3861 server_base.cc:1034] running on GCE node
I20260926 20:42:41.396034 3861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:41.397439 3861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:41.398708 3861 hybrid_clock.cc:648] HybridClock initialized: now 1790455361398637 us; error 71 us; skew 500 ppm
I20260926 20:42:41.399128 3861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:41 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455361, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:41.403685 3861 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:42:41.405741 3861 webserver.cc:460] Webserver started at http://127.23.44.194:36827/ using document root <none> and password file <none>
I20260926 20:42:41.406653 3861 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:41.406788 3861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:42:41.407121 3861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:42:41 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455361, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:42:41.424054 3861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "407ec77a2a2843019749932d38471fca"
format_stamp: "Formatted at 2026-09-26 20:42:41 on dist-test-slave-4d5t"
server_key: "49d0565613c0c5ced4a1487903ff52c7"
server_key_iv: "2bb5d61acb87fac40a308f6a307a3373"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:41.425108 3861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "407ec77a2a2843019749932d38471fca"
format_stamp: "Formatted at 2026-09-26 20:42:41 on dist-test-slave-4d5t"
server_key: "49d0565613c0c5ced4a1487903ff52c7"
server_key_iv: "2bb5d61acb87fac40a308f6a307a3373"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:41.432509 3861 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260926 20:42:41.453868 3884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:41.455483 3861 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260926 20:42:41.455675 3861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "407ec77a2a2843019749932d38471fca"
format_stamp: "Formatted at 2026-09-26 20:42:41 on dist-test-slave-4d5t"
server_key: "49d0565613c0c5ced4a1487903ff52c7"
server_key_iv: "2bb5d61acb87fac40a308f6a307a3373"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:41.455893 3861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:41.469872 3861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:41.474184 3861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:41.474505 3861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:41.475574 3861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:42:41.477054 3861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:42:41.477128 3861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:41.477265 3861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:42:41.477317 3861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:41.520272 3861 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:38287
I20260926 20:42:41.520289 3996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:38287 every 8 connection(s)
I20260926 20:42:41.521818 3861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:42:41.525310 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 3861
I20260926 20:42:41.525475 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:42:41.526129 23731 external_mini_cluster.cc:1428] Setting key 63fa7c7c39eaefe4fe8b625329d578ed
Sep 26 20:42:41 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455361, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:41.549139 3999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:41.524808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:41705 (local address 127.23.44.254:34643)
0926 20:42:41.529022 (+ 4214us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:41.529039 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:41.529067 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:42:41.530481 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:41.530486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:41.530561 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:41.530665 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:41.532846 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:41.534093 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:41.536399 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:41.537162 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:41.537308 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:41.541698 (+ 4390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:41.541705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:41.541709 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:41.541750 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:41.545262 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:41.546335 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:41.546341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:41.546345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:41.546494 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:41.547025 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:41.547030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:41.547034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:41.547392 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:41.547549 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:41.548498 (+ 949us) server_negotiation.cc:299] Negotiation successful
0926 20:42:41.548759 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4052,"thread_start_us":3947,"threads_started":1}
I20260926 20:42:41.550648 4000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:41.525442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:34643 (local address 127.23.44.194:41705)
0926 20:42:41.527833 (+ 2391us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:41.527894 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:41.530184 (+ 2290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:41.531133 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:41.531145 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:41.531761 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:41.532631 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:41.532657 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:41.534271 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:41.534276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:41.536210 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:41.536241 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:41.537424 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:41.537430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:41.537727 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:41.538547 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:41.538576 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:41.541409 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:41.545436 (+ 4027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:41.545455 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:41.545479 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:41.546186 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:41.546621 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:41.546629 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:41.546635 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:41.546883 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:41.547535 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:41.547546 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:41.548038 (+ 492us) client_negotiation.cc:769] Sending connection context
0926 20:42:41.548361 (+ 323us) client_negotiation.cc:240] Negotiation successful
0926 20:42:41.549020 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2075,"thread_start_us":1855,"threads_started":1}
I20260926 20:42:41.552198 3997 heartbeater.cc:347] Connected to a master server at 127.23.44.254:34643
I20260926 20:42:41.552598 3997 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
I20260926 20:42:41.553535 3997 heartbeater.cc:511] Master 127.23.44.254:34643 requested a full tablet report, sending...
I20260926 20:42:41.555322 3641 ts_manager.cc:194] Registered new tserver with Master: 407ec77a2a2843019749932d38471fca (127.23.44.194:38287)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
I20260926 20:42:41.556648 3641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:41705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:42:41.587622 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.44.212:35893/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:34643
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:42:41.874717 3854 heartbeater.cc:503] Master 127.23.44.254:34643 was elected leader, sending a full tablet report...
W20260926 20:42:41.937849 4003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:41.938182 4003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:41.938357 4003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:41.946003 4003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:41.946113 4003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:41.946275 4003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:42:41.956508 4003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.44.212:35893/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:34643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:41.957870 4003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:41.959213 4003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:41.975028 4009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:41.975085 4011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:41.975101 4008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:41.976070 4003 server_base.cc:1034] running on GCE node
I20260926 20:42:41.976771 4003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:41.977891 4003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:41.979210 4003 hybrid_clock.cc:648] HybridClock initialized: now 1790455361979082 us; error 80 us; skew 500 ppm
I20260926 20:42:41.979707 4003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:41 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455361, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:41.984218 4003 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:42:41.986135 4003 webserver.cc:460] Webserver started at http://127.23.44.195:44191/ using document root <none> and password file <none>
I20260926 20:42:41.986866 4003 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:41.986984 4003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:42:41.987248 4003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 20:42:41 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455361, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:42:42.003335 4003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "25401a5905074a66bef6fc515a45d289"
format_stamp: "Formatted at 2026-09-26 20:42:41 on dist-test-slave-4d5t"
server_key: "246f0024893668b4cc679ae6f591eb86"
server_key_iv: "1fc9fe6e7c32f2f2c5fc4c2d024f713b"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:42.004508 4003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "25401a5905074a66bef6fc515a45d289"
format_stamp: "Formatted at 2026-09-26 20:42:41 on dist-test-slave-4d5t"
server_key: "246f0024893668b4cc679ae6f591eb86"
server_key_iv: "1fc9fe6e7c32f2f2c5fc4c2d024f713b"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:42.012655 4003 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260926 20:42:42.031859 4019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:42.033953 4003 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260926 20:42:42.034169 4003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "25401a5905074a66bef6fc515a45d289"
format_stamp: "Formatted at 2026-09-26 20:42:41 on dist-test-slave-4d5t"
server_key: "246f0024893668b4cc679ae6f591eb86"
server_key_iv: "1fc9fe6e7c32f2f2c5fc4c2d024f713b"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:42.034472 4003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:42.062947 4003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:42.067528 4003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:42.067884 4003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:42.068985 4003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:42:42.070379 4003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:42:42.070451 4003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:42.070564 4003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:42:42.070624 4003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:42.110183 4003 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:40747
I20260926 20:42:42.110230 4131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:40747 every 8 connection(s)
I20260926 20:42:42.111644 4003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:42:42.113988 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 4003
I20260926 20:42:42.114259 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:42:42.115026 23731 external_mini_cluster.cc:1428] Setting key 0e452a0ea31c429ee64db0ccdfbbc1ac
Sep 26 20:42:42 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455361, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:42.134627 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:42.114753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:50347 (local address 127.23.44.254:34643)
0926 20:42:42.115478 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:42.115486 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:42.116785 (+ 1299us) server_negotiation.cc:407] Connection header received
0926 20:42:42.119374 (+ 2589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:42.119380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:42.119488 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:42.119597 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:42.121279 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:42.122013 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:42.123591 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:42.124030 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:42.124131 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:42.127882 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:42.127887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:42.127891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:42.127925 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:42.131117 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:42.132213 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:42.132219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:42.132222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:42.132363 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:42.132933 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:42.132937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:42.132940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:42.133228 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:42.133352 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:42.134190 (+ 838us) server_negotiation.cc:299] Negotiation successful
0926 20:42:42.134360 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":320,"threads_started":1}
I20260926 20:42:42.135649 4135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:42.115250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:34643 (local address 127.23.44.195:50347)
0926 20:42:42.116500 (+ 1250us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:42.116613 (+ 113us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:42.119068 (+ 2455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:42.119896 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:42.119905 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:42.120371 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:42.121098 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:42.121132 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:42.122187 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:42.122191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:42.123429 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:42.123448 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:42.124243 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:42.124249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:42.124483 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:42.125331 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:42.125360 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:42.127635 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:42.131293 (+ 3658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:42.131303 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:42.131325 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:42.132065 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:42.132505 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:42.132509 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:42.132512 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:42.132776 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:42.133361 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:42.133367 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:42.133765 (+ 398us) client_negotiation.cc:769] Sending connection context
0926 20:42:42.134055 (+ 290us) client_negotiation.cc:240] Negotiation successful
0926 20:42:42.134475 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":533,"threads_started":1}
I20260926 20:42:42.137095 4132 heartbeater.cc:347] Connected to a master server at 127.23.44.254:34643
I20260926 20:42:42.137408 4132 heartbeater.cc:464] Registering TS with master...
I20260926 20:42:42.138124 4132 heartbeater.cc:511] Master 127.23.44.254:34643 requested a full tablet report, sending...
I20260926 20:42:42.139727 3641 ts_manager.cc:194] Registered new tserver with Master: 25401a5905074a66bef6fc515a45d289 (127.23.44.195:40747)
I20260926 20:42:42.140913 3641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:50347
I20260926 20:42:42.145074 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:42:42.146757 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 3721
Sep 26 20:42:42 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:42:42.165931 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:42.147813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49660 (local address 127.23.44.193:41347)
0926 20:42:42.148638 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:42.148650 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:42.148772 (+ 122us) server_negotiation.cc:407] Connection header received
0926 20:42:42.148856 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:42.148862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:42.149074 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:42.149253 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:42.150771 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:42.151813 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:42.153295 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:42.153885 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:42.154029 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:42.158958 (+ 4929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:42.158975 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:42.158994 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:42.159035 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:42.162398 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:42.163395 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:42.163404 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:42.163407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:42.163567 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:42.164196 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:42.164200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:42.164203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:42.164709 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:42.164983 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:42.165391 (+ 408us) server_negotiation.cc:299] Negotiation successful
0926 20:42:42.165618 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":433,"threads_started":1}
I20260926 20:42:42.167388 3849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:42.254093 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 3721
I20260926 20:42:42.271065 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 3861
Sep 26 20:42:42 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:42:42.291102 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:42.272284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45048 (local address 127.23.44.194:38287)
0926 20:42:42.273091 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:42.273101 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:42.273136 (+ 35us) server_negotiation.cc:407] Connection header received
0926 20:42:42.273398 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:42.273409 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:42.273724 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:42.273912 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:42.275367 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:42.276800 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:42.278544 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:42.279197 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:42.279336 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:42.284446 (+ 5110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:42.284461 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:42.284479 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:42.284525 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:42.288010 (+ 3485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:42.288857 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:42.288865 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:42.288867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:42.289014 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:42.289431 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:42.289435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:42.289438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:42.289801 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:42.290103 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:42.290423 (+ 320us) server_negotiation.cc:299] Negotiation successful
0926 20:42:42.290713 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":468,"threads_started":1}
I20260926 20:42:42.292415 3992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:42.381155 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 3861
I20260926 20:42:42.398898 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 4003
Sep 26 20:42:42 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:42:42.417239 4141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:42.400266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56788 (local address 127.23.44.195:40747)
0926 20:42:42.401062 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:42.401069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:42.401116 (+ 47us) server_negotiation.cc:407] Connection header received
0926 20:42:42.401185 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:42.401192 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:42.401406 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:42.401540 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:42.402986 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:42.404017 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:42.405705 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:42.406329 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:42.406433 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:42.411035 (+ 4602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:42.411050 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:42.411067 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:42.411107 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:42.414102 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:42.415036 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:42.415046 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:42.415049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:42.415198 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:42.415590 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:42.415593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:42.415595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:42.415959 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:42.416218 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:42.416527 (+ 309us) server_negotiation.cc:299] Negotiation successful
0926 20:42:42.416839 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":449,"threads_started":1}
I20260926 20:42:42.418824 4127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:42.505899 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 4003
I20260926 20:42:42.521920 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 3609
I20260926 20:42:42.523191 3671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:42.635273 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 3609
I20260926 20:42:42.664881 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:34643
--webserver_interface=127.23.44.254
--webserver_port=43573
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:34643
--ranger_config_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.44.212:35893/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.44.212:35893/kms with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:42:43.029330 4144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:43.029763 4144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:43.029949 4144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:43.039153 4144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:42:43.039306 4144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:43.039356 4144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:43.039489 4144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:42:43.039582 4144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:42:43.050132 4144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.44.212:35893/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:34643
--ranger_config_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:34643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=43573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:43.051990 4144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:43.053776 4144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:43.069048 4149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:43.069031 4150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:43.068833 4152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:43.070588 4144 server_base.cc:1034] running on GCE node
I20260926 20:42:43.071452 4144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:43.073079 4144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:43.074335 4144 hybrid_clock.cc:648] HybridClock initialized: now 1790455363074269 us; error 59 us; skew 500 ppm
I20260926 20:42:43.074712 4144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:43 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455363, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:43.078929 4144 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:42:43.081254 4144 webserver.cc:460] Webserver started at http://127.23.44.254:43573/ using document root <none> and password file <none>
I20260926 20:42:43.081985 4144 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:43.082147 4144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 20:42:43 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455363, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:42:43.145843 4144 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:42:43.149097 4160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:43.150657 4144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260926 20:42:43.150862 4144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "c942f3fa1b3949388f7a6569e3206161"
format_stamp: "Formatted at 2026-09-26 20:42:38 on dist-test-slave-4d5t"
server_key: "1811b732329af39f4008adfa6bab2cd4"
server_key_iv: "bf8ec87b7e3ef22289d7f9f55f9e4829"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:43.151889 4144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:43.166798 4144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:43.171491 4144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:43.171775 4144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:43.192615 4144 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:34643
I20260926 20:42:43.192759 4211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:34643 every 8 connection(s)
I20260926 20:42:43.194218 4144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:42:43.201655 4212 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:42:43.203575 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 4144
I20260926 20:42:43.204617 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.44.212:35893/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:41347
--local_ip_for_outbound_sockets=127.23.44.193
--tserver_master_addrs=127.23.44.254:34643
--webserver_port=41149
--webserver_interface=127.23.44.193
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.44.212:35893/kms with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:42:43.206326 4212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161: Bootstrap starting.
I20260926 20:42:43.227589 4212 log.cc:824] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161: Log is configured to *not* fsync() on all Append() calls
I20260926 20:42:43.236670 4212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:42:43.237211 4212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161: Bootstrap complete.
I20260926 20:42:43.246553 4212 raft_consensus.cc:348] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c942f3fa1b3949388f7a6569e3206161" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34643 } }
I20260926 20:42:43.247416 4212 raft_consensus.cc:729] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c942f3fa1b3949388f7a6569e3206161, State: Initialized, Role: FOLLOWER
I20260926 20:42:43.248139 4212 consensus_queue.cc:260] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c942f3fa1b3949388f7a6569e3206161" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34643 } }
I20260926 20:42:43.248376 4212 raft_consensus.cc:388] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:42:43.248492 4212 raft_consensus.cc:482] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:42:43.248714 4212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 1 FOLLOWER]: Advancing to term 2
I20260926 20:42:43.252628 4212 raft_consensus.cc:504] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c942f3fa1b3949388f7a6569e3206161" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34643 } }
I20260926 20:42:43.253168 4212 leader_election.cc:302] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c942f3fa1b3949388f7a6569e3206161; no voters:
I20260926 20:42:43.253815 4212 leader_election.cc:288] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 20:42:43.253973 4216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 20:42:43.255436 4216 raft_consensus.cc:686] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [term 2 LEADER]: Becoming Leader. State: Replica: c942f3fa1b3949388f7a6569e3206161, State: Running, Role: LEADER
I20260926 20:42:43.256199 4212 sys_catalog.cc:564] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:42:43.256008 4216 consensus_queue.cc:237] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c942f3fa1b3949388f7a6569e3206161" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34643 } }
I20260926 20:42:43.258857 4218 sys_catalog.cc:455] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c942f3fa1b3949388f7a6569e3206161. Latest consensus state: current_term: 2 leader_uuid: "c942f3fa1b3949388f7a6569e3206161" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c942f3fa1b3949388f7a6569e3206161" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34643 } } }
I20260926 20:42:43.259444 4218 sys_catalog.cc:458] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [sys.catalog]: This master's current role is: LEADER
I20260926 20:42:43.259399 4217 sys_catalog.cc:455] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c942f3fa1b3949388f7a6569e3206161" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c942f3fa1b3949388f7a6569e3206161" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 34643 } } }
I20260926 20:42:43.259735 4217 sys_catalog.cc:458] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161 [sys.catalog]: This master's current role is: LEADER
I20260926 20:42:43.275169 4221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:42:43.281332 4221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:42:43.282861 4221 catalog_manager.cc:1259] Loaded cluster ID: 8f6e23b04a3b4edc99fb9bfbbffbfa4c
I20260926 20:42:43.283042 4221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:42:43.287154 4221 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:42:43.288923 4221 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161: Loaded TSK: 0
I20260926 20:42:43.290650 4221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:42:43.297055 4212 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260926 20:42:43.662693 4214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:43.663144 4214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:43.663312 4214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:43.677691 4214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:43.677879 4214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:43.678095 4214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:42:43.696165 4214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.44.212:35893/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:41347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=41149
--tserver_master_addrs=127.23.44.254:34643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:43.697906 4214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:43.699620 4214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:43.725076 4251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:43.728149 4254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:43.728729 4250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:43.729550 4214 server_base.cc:1034] running on GCE node
I20260926 20:42:43.730600 4214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:43.738555 4214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:43.740895 4214 hybrid_clock.cc:648] HybridClock initialized: now 1790455363740813 us; error 115 us; skew 500 ppm
I20260926 20:42:43.741402 4214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:43 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455363, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:43.750073 4214 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:42:43.755028 4214 webserver.cc:460] Webserver started at http://127.23.44.193:41149/ using document root <none> and password file <none>
I20260926 20:42:43.756157 4214 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:43.756333 4214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 20:42:43 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455363, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:42:43.787874 4214 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260926 20:42:43.792865 4261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:43.795437 4214 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:42:43.795702 4214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "4d435588d72d4b3caa9a473404d634d6"
format_stamp: "Formatted at 2026-09-26 20:42:40 on dist-test-slave-4d5t"
server_key: "f2618d75c4f8d1a9dafe4b9e5c5b0809"
server_key_iv: "a735e6a30d57462d3bed38a1a72ee3e4"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:43.797180 4214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:43.817600 4214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:43.823374 4214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:43.823720 4214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:43.837172 4214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:42:43.845633 4214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:42:43.845785 4214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:43.845916 4214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:42:43.846007 4214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:43.922646 4214 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:41347
I20260926 20:42:43.922691 4373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:41347 every 8 connection(s)
I20260926 20:42:43.924173 4214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:42:43.929665 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 4214
I20260926 20:42:43.930704 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.44.212:35893/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:38287
--local_ip_for_outbound_sockets=127.23.44.194
--tserver_master_addrs=127.23.44.254:34643
--webserver_port=36827
--webserver_interface=127.23.44.194
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.44.212:35893/kms with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:42:43 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455363, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:43.963299 4376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:43.928034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:46547 (local address 127.23.44.254:34643)
0926 20:42:43.932838 (+ 4804us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:43.932859 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:43.936987 (+ 4128us) server_negotiation.cc:407] Connection header received
0926 20:42:43.937457 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:43.937474 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:43.937929 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:43.938324 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:43.940263 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:43.942429 (+ 2166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:43.945024 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:43.946083 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:43.946235 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:43.952504 (+ 6269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:43.952516 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:43.952537 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:43.952619 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:43.957209 (+ 4590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:43.958429 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:43.958439 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:43.958446 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:43.958668 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:43.959228 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:43.959234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:43.959237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:43.959970 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:43.960200 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:43.961355 (+ 1155us) server_negotiation.cc:299] Negotiation successful
0926 20:42:43.961776 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4478,"thread_start_us":4223,"threads_started":1}
I20260926 20:42:43.963374 4377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:43.928309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:34643 (local address 127.23.44.193:46547)
0926 20:42:43.933511 (+ 5202us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:43.933571 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:43.936436 (+ 2865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:43.938521 (+ 2085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:43.938533 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:43.939202 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:43.940003 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:43.940036 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:43.942659 (+ 2623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:43.942666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:43.944803 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:43.944832 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:43.946725 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:43.946733 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:43.947020 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:43.947835 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:43.947867 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:43.952088 (+ 4221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:43.957454 (+ 5366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:43.957468 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:43.957488 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:43.958222 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:43.958809 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:43.958816 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:43.958823 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:43.959070 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:43.960138 (+ 1068us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:43.960150 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:43.960710 (+ 560us) client_negotiation.cc:769] Sending connection context
0926 20:42:43.961051 (+ 341us) client_negotiation.cc:240] Negotiation successful
0926 20:42:43.961744 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4887,"thread_start_us":8261,"threads_started":1}
I20260926 20:42:43.968827 4374 heartbeater.cc:347] Connected to a master server at 127.23.44.254:34643
I20260926 20:42:43.969187 4374 heartbeater.cc:464] Registering TS with master...
I20260926 20:42:43.970139 4374 heartbeater.cc:511] Master 127.23.44.254:34643 requested a full tablet report, sending...
W20260926 20:42:43.973754 4374 heartbeater.cc:644] Failed to heartbeat to 127.23.44.254:34643 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 20:42:43.974009 4374 heartbeater.cc:464] Registering TS with master...
I20260926 20:42:43.974311 4374 heartbeater.cc:511] Master 127.23.44.254:34643 requested a full tablet report, sending...
I20260926 20:42:43.975696 4374 heartbeater.cc:464] Registering TS with master...
I20260926 20:42:43.976020 4374 heartbeater.cc:511] Master 127.23.44.254:34643 requested a full tablet report, sending...
W20260926 20:42:43.977320 4374 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 20:42:44.376215 4379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:44.376549 4379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:44.376760 4379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:44.384454 4379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:44.384649 4379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:44.384846 4379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:42:44.394975 4379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.44.212:35893/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:38287
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=36827
--tserver_master_addrs=127.23.44.254:34643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:44.396422 4379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:44.397769 4379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:44.415176 4391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:44.415294 4392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:44.415864 4394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:44.416944 4379 server_base.cc:1034] running on GCE node
I20260926 20:42:44.417618 4379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:44.418721 4379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:44.420048 4379 hybrid_clock.cc:648] HybridClock initialized: now 1790455364419969 us; error 58 us; skew 500 ppm
I20260926 20:42:44.420435 4379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:44 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455364, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:44.424176 4379 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:42:44.429301 4379 webserver.cc:460] Webserver started at http://127.23.44.194:36827/ using document root <none> and password file <none>
I20260926 20:42:44.430060 4379 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:44.430229 4379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 20:42:44 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455364, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:42:44.461926 4379 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.004s sys 0.008s
I20260926 20:42:44.466638 4404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:44.468711 4379 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:42:44.468986 4379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "407ec77a2a2843019749932d38471fca"
format_stamp: "Formatted at 2026-09-26 20:42:41 on dist-test-slave-4d5t"
server_key: "49d0565613c0c5ced4a1487903ff52c7"
server_key_iv: "2bb5d61acb87fac40a308f6a307a3373"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:44.470518 4379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:44.497715 4379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:44.503456 4379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:44.503839 4379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:44.505172 4379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:42:44.506768 4379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:42:44.506896 4379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:44.507046 4379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:42:44.507135 4379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:44.596304 4379 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:38287
I20260926 20:42:44.596387 4518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:38287 every 8 connection(s)
I20260926 20:42:44.598474 4379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:42:44.603730 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 4379
I20260926 20:42:44.604471 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.44.212:35893/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:40747
--local_ip_for_outbound_sockets=127.23.44.195
--tserver_master_addrs=127.23.44.254:34643
--webserver_port=44191
--webserver_interface=127.23.44.195
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.44.212:35893/kms with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:42:44 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455364, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:44.634579 4522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:44.603279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:53603 (local address 127.23.44.254:34643)
0926 20:42:44.615215 (+ 11936us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:44.615223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:44.615250 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:42:44.615328 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:44.615333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:44.615428 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:44.615583 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:44.618179 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:44.619402 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:44.621984 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:44.622703 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:44.622835 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:44.627443 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:44.627451 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:44.627456 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:44.627498 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:44.631167 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:44.632069 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:44.632073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:44.632075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:44.632199 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:44.632718 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:44.632722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:44.632724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:44.633005 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:44.633174 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:44.633925 (+ 751us) server_negotiation.cc:299] Negotiation successful
0926 20:42:44.634159 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11776,"thread_start_us":12435,"threads_started":1}
I20260926 20:42:44.635752 4521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:44.602280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:34643 (local address 127.23.44.194:53603)
0926 20:42:44.603765 (+ 1485us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:44.603813 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:44.609075 (+ 5262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:44.616236 (+ 7161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:44.616253 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:44.616979 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:44.617900 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:44.617941 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:44.619607 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:44.619614 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:44.621762 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:44.621780 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:44.623178 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:44.623186 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:44.623501 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:44.624384 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:44.624427 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:44.627157 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:44.631345 (+ 4188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:44.631355 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:44.631376 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:44.631917 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:44.632326 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:44.632331 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:44.632335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:44.632533 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:44.633148 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:44.633157 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:44.633558 (+ 401us) client_negotiation.cc:769] Sending connection context
0926 20:42:44.633831 (+ 273us) client_negotiation.cc:240] Negotiation successful
0926 20:42:44.634461 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1164,"thread_start_us":1023,"threads_started":1}
I20260926 20:42:44.638092 4519 heartbeater.cc:347] Connected to a master server at 127.23.44.254:34643
I20260926 20:42:44.638590 4519 heartbeater.cc:464] Registering TS with master...
I20260926 20:42:44.640002 4519 heartbeater.cc:511] Master 127.23.44.254:34643 requested a full tablet report, sending...
I20260926 20:42:44.642612 4177 ts_manager.cc:194] Registered new tserver with Master: 407ec77a2a2843019749932d38471fca (127.23.44.194:38287)
I20260926 20:42:44.644403 4177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:53603
Sep 26 20:42:44 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790455364, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:44.977458 4374 heartbeater.cc:464] Registering TS with master...
I20260926 20:42:44.977955 4374 heartbeater.cc:511] Master 127.23.44.254:34643 requested a full tablet report, sending...
I20260926 20:42:44.979334 4177 ts_manager.cc:194] Registered new tserver with Master: 4d435588d72d4b3caa9a473404d634d6 (127.23.44.193:41347)
I20260926 20:42:44.980947 4177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:46547
W20260926 20:42:45.058976 4523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:45.059476 4523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:45.059674 4523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:45.071697 4523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:45.072021 4523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:45.072345 4523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:42:45.088528 4523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.44.212:35893/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:40747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=44191
--tserver_master_addrs=127.23.44.254:34643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:45.090349 4523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:45.092167 4523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:45.113143 4529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:45.117316 4530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:45.120268 4532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:45.122802 4523 server_base.cc:1034] running on GCE node
I20260926 20:42:45.126446 4523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:45.133471 4523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:45.134888 4523 hybrid_clock.cc:648] HybridClock initialized: now 1790455365134780 us; error 113 us; skew 500 ppm
I20260926 20:42:45.135581 4523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:45 dist-test-slave-4d5t krb5kdc[3165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455365, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:45.140015 4523 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:42:45.142159 4523 webserver.cc:460] Webserver started at http://127.23.44.195:44191/ using document root <none> and password file <none>
I20260926 20:42:45.143059 4523 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:45.143203 4523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 20:42:45 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455364, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
Sep 26 20:42:45 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455365, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:42:45.199637 4523 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.006s sys 0.007s
I20260926 20:42:45.205107 4540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:45.207176 4523 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260926 20:42:45.207448 4523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "25401a5905074a66bef6fc515a45d289"
format_stamp: "Formatted at 2026-09-26 20:42:41 on dist-test-slave-4d5t"
server_key: "246f0024893668b4cc679ae6f591eb86"
server_key_iv: "1fc9fe6e7c32f2f2c5fc4c2d024f713b"
server_key_version: "kuduclusterkey@0"
I20260926 20:42:45.209051 4523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:45.253175 4523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:45.258642 4523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:45.258991 4523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:45.260339 4523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:42:45.262367 4523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:42:45.262501 4523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:45.262599 4523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:42:45.262681 4523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:45.316303 4523 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:40747
I20260926 20:42:45.316385 4652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:40747 every 8 connection(s)
I20260926 20:42:45.318329 4523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:42:45.322042 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 4523
Sep 26 20:42:45 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455365, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:45.367663 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.322776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:55495 (local address 127.23.44.254:34643)
0926 20:42:45.329042 (+ 6266us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:45.329051 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:45.332624 (+ 3573us) server_negotiation.cc:407] Connection header received
0926 20:42:45.334543 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:45.334550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:45.334645 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:45.334769 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:45.337168 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.344363 (+ 7195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.352441 (+ 8078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.353207 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.353333 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.359159 (+ 5826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:45.359169 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:45.359175 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:45.359230 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:45.363274 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:45.364496 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:45.364502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:45.364506 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:45.364692 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:45.365302 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:45.365308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:45.365311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:45.365716 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:45.365846 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:45.366929 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0926 20:42:45.367190 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6097,"thread_start_us":5965,"threads_started":1}
I20260926 20:42:45.370494 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.322104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:34643 (local address 127.23.44.195:55495)
0926 20:42:45.329696 (+ 7592us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:45.329761 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:45.334131 (+ 4370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:45.335255 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:45.335269 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:45.335932 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:45.336926 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:45.336955 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.349635 (+ 12680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.349644 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:45.352142 (+ 2498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:45.352189 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.353855 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.353865 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:45.354269 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.355412 (+ 1143us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:45.355459 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:45.358784 (+ 3325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:45.363612 (+ 4828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:45.363623 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:45.363644 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:45.364323 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:45.364858 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:45.364865 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:45.364869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:45.365157 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:45.365962 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:45.365972 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:45.366462 (+ 490us) client_negotiation.cc:769] Sending connection context
0926 20:42:45.368334 (+ 1872us) client_negotiation.cc:240] Negotiation successful
0926 20:42:45.368951 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7262,"thread_start_us":7024,"threads_started":1}
I20260926 20:42:45.373198 4653 heartbeater.cc:347] Connected to a master server at 127.23.44.254:34643
I20260926 20:42:45.373594 4653 heartbeater.cc:464] Registering TS with master...
I20260926 20:42:45.377068 4657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.323161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40034 (local address 127.23.44.254:34643)
0926 20:42:45.333125 (+ 9964us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:45.333133 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:45.335152 (+ 2019us) server_negotiation.cc:407] Connection header received
0926 20:42:45.335428 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:45.335433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:45.335529 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:45.335603 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:45.338020 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.350419 (+ 12399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.362926 (+ 12507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.363672 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.363777 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.367617 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:45.367625 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:45.367631 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:45.367671 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:45.371212 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:45.374058 (+ 2846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:45.374064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:45.374068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:45.374230 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:45.375081 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:45.375086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:45.375090 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:45.375481 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:45.375761 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:45.376368 (+ 607us) server_negotiation.cc:299] Negotiation successful
0926 20:42:45.376674 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9812,"thread_start_us":9688,"threads_started":1}
I20260926 20:42:45.378816 4653 heartbeater.cc:511] Master 127.23.44.254:34643 requested a full tablet report, sending...
I20260926 20:42:45.380858 4177 ts_manager.cc:194] Registered new tserver with Master: 25401a5905074a66bef6fc515a45d289 (127.23.44.195:40747)
I20260926 20:42:45.382085 4177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:55495
I20260926 20:42:45.384936 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:42:45.458248 4657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.431009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40042 (local address 127.23.44.254:34643)
0926 20:42:45.431222 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:45.431229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:45.437264 (+ 6035us) server_negotiation.cc:407] Connection header received
0926 20:42:45.437553 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:45.437563 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:45.437689 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:45.437778 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:45.440373 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.441780 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.444311 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.445170 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.445292 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.449797 (+ 4505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:45.449806 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:45.449814 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:45.449875 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:45.454176 (+ 4301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:45.455281 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:45.455290 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:45.455296 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:45.455486 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:45.456024 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:45.456031 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:45.456037 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:45.456483 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:45.456807 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:45.457484 (+ 677us) server_negotiation.cc:299] Negotiation successful
0926 20:42:45.457787 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 20:42:45.467422 4177 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:42:45.483990 4177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Sep 26 20:42:45 dist-test-slave-4d5t krb5kdc[3165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455364, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for HTTP/127.23.44.212@KRBTEST.COM
I20260926 20:42:45.588495 4674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.528837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:40747 (local address 127.0.0.1:56392)
0926 20:42:45.571489 (+ 42652us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:45.571553 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:45.571801 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:45.574109 (+ 2308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:45.574115 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:45.574142 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:42:45.574657 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:45.574671 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.579785 (+ 5114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.579791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:45.582691 (+ 2900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:45.582712 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.586965 (+ 4253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.586973 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:45.587227 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.587301 (+ 74us) client_negotiation.cc:769] Sending connection context
0926 20:42:45.587440 (+ 139us) client_negotiation.cc:240] Negotiation successful
0926 20:42:45.587681 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42287,"thread_start_us":50545,"threads_started":1}
I20260926 20:42:45.591034 4670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.528716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56392 (local address 127.23.44.195:40747)
0926 20:42:45.570929 (+ 42213us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:45.570941 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:45.572840 (+ 1899us) server_negotiation.cc:407] Connection header received
0926 20:42:45.573022 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:45.573032 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:45.573302 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:45.574901 (+ 1599us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:42:45.575255 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.579320 (+ 4065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.584111 (+ 4791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.586027 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.590038 (+ 4011us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.590331 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:45.590429 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 20:42:45.590614 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41957,"thread_start_us":41742,"threads_started":1}
I20260926 20:42:45.595553 4588 tablet_service.cc:1467] Processing CreateTablet for tablet 4daa31fc49d441628bcadf5d34da25e0 (DEFAULT_TABLE table=test-table [id=1d427b2e9cb0460a8313b068ed7115ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:42:45.598969 4588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4daa31fc49d441628bcadf5d34da25e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:42:45.627825 4671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.542309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:38287 (local address 127.0.0.1:51654)
0926 20:42:45.573526 (+ 31217us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:45.573546 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:45.581424 (+ 7878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:45.583166 (+ 1742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:45.583174 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:45.583212 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:42:45.583880 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:45.583897 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.586564 (+ 2667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.586573 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:45.589232 (+ 2659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:45.589252 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.626641 (+ 37389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.626652 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:45.626940 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.627021 (+ 81us) client_negotiation.cc:769] Sending connection context
0926 20:42:45.627127 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 20:42:45.627324 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31072,"thread_start_us":28681,"threads_started":1}
I20260926 20:42:45.629415 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.542573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51654 (local address 127.23.44.194:38287)
0926 20:42:45.554309 (+ 11736us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:45.554324 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:45.573660 (+ 19336us) server_negotiation.cc:407] Connection header received
0926 20:42:45.581778 (+ 8118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:45.581800 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:45.582265 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:45.582526 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:42:45.584228 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.586273 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.617647 (+ 31374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.624306 (+ 6659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.628192 (+ 3886us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.628575 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:45.628677 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 20:42:45.628925 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11469,"thread_start_us":11235,"threads_started":1}
I20260926 20:42:45.634789 4454 tablet_service.cc:1467] Processing CreateTablet for tablet 4daa31fc49d441628bcadf5d34da25e0 (DEFAULT_TABLE table=test-table [id=1d427b2e9cb0460a8313b068ed7115ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:42:45.636736 4454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4daa31fc49d441628bcadf5d34da25e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:42:45.637563 4676 tablet_bootstrap.cc:492] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289: Bootstrap starting.
I20260926 20:42:45.642045 4672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.532902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:41347 (local address 127.0.0.1:38014)
0926 20:42:45.572495 (+ 39593us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:45.572517 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:45.572956 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:45.573786 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:45.573796 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:45.573834 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:42:45.574384 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:45.574399 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.577175 (+ 2776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.577183 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:45.580945 (+ 3762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:45.580990 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.640747 (+ 59757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.640758 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:45.641077 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.641160 (+ 83us) client_negotiation.cc:769] Sending connection context
0926 20:42:45.641302 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 20:42:45.641503 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39427,"mutex_wait_us":2,"thread_start_us":39266,"threads_started":1}
I20260926 20:42:45.643100 4673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.533215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38014 (local address 127.23.44.193:41347)
0926 20:42:45.556022 (+ 22807us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:45.556033 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:45.572709 (+ 16676us) server_negotiation.cc:407] Connection header received
0926 20:42:45.573348 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:45.573358 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:45.573577 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:45.573779 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:42:45.574638 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.576959 (+ 2321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.618485 (+ 41526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.622457 (+ 3972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.634080 (+ 11623us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.634492 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:45.642383 (+ 7891us) server_negotiation.cc:299] Negotiation successful
0926 20:42:45.642645 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22594,"thread_start_us":22404,"threads_started":1}
I20260926 20:42:45.643819 4676 tablet_bootstrap.cc:654] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289: Neither blocks nor log segments found. Creating new log.
I20260926 20:42:45.646011 4676 log.cc:824] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289: Log is configured to *not* fsync() on all Append() calls
I20260926 20:42:45.648180 4309 tablet_service.cc:1467] Processing CreateTablet for tablet 4daa31fc49d441628bcadf5d34da25e0 (DEFAULT_TABLE table=test-table [id=1d427b2e9cb0460a8313b068ed7115ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:42:45.648818 4519 heartbeater.cc:503] Master 127.23.44.254:34643 was elected leader, sending a full tablet report...
I20260926 20:42:45.656082 4309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4daa31fc49d441628bcadf5d34da25e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:42:45.676805 4676 tablet_bootstrap.cc:492] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289: No bootstrap required, opened a new log
I20260926 20:42:45.677310 4676 ts_tablet_manager.cc:1397] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289: Time spent bootstrapping tablet: real 0.040s user 0.020s sys 0.014s
I20260926 20:42:45.690845 4676 raft_consensus.cc:348] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25401a5905074a66bef6fc515a45d289" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40747 } } peers { permanent_uuid: "4d435588d72d4b3caa9a473404d634d6" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41347 } } peers { permanent_uuid: "407ec77a2a2843019749932d38471fca" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38287 } }
I20260926 20:42:45.691471 4676 raft_consensus.cc:374] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:42:45.691604 4676 raft_consensus.cc:729] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25401a5905074a66bef6fc515a45d289, State: Initialized, Role: FOLLOWER
I20260926 20:42:45.692489 4676 consensus_queue.cc:260] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25401a5905074a66bef6fc515a45d289" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40747 } } peers { permanent_uuid: "4d435588d72d4b3caa9a473404d634d6" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41347 } } peers { permanent_uuid: "407ec77a2a2843019749932d38471fca" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38287 } }
I20260926 20:42:45.694192 4678 tablet_bootstrap.cc:492] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6: Bootstrap starting.
I20260926 20:42:45.703408 4678 tablet_bootstrap.cc:654] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6: Neither blocks nor log segments found. Creating new log.
I20260926 20:42:45.713274 4678 log.cc:824] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6: Log is configured to *not* fsync() on all Append() calls
I20260926 20:42:45.713454 4653 heartbeater.cc:503] Master 127.23.44.254:34643 was elected leader, sending a full tablet report...
I20260926 20:42:45.714397 4676 ts_tablet_manager.cc:1428] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289: Time spent starting tablet: real 0.037s user 0.021s sys 0.016s
I20260926 20:42:45.718294 4678 tablet_bootstrap.cc:492] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6: No bootstrap required, opened a new log
I20260926 20:42:45.718643 4678 ts_tablet_manager.cc:1397] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.003s
I20260926 20:42:45.733153 4678 raft_consensus.cc:348] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25401a5905074a66bef6fc515a45d289" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40747 } } peers { permanent_uuid: "4d435588d72d4b3caa9a473404d634d6" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41347 } } peers { permanent_uuid: "407ec77a2a2843019749932d38471fca" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38287 } }
I20260926 20:42:45.733804 4678 raft_consensus.cc:374] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:42:45.733925 4678 raft_consensus.cc:729] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d435588d72d4b3caa9a473404d634d6, State: Initialized, Role: FOLLOWER
I20260926 20:42:45.734925 4678 consensus_queue.cc:260] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25401a5905074a66bef6fc515a45d289" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40747 } } peers { permanent_uuid: "4d435588d72d4b3caa9a473404d634d6" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41347 } } peers { permanent_uuid: "407ec77a2a2843019749932d38471fca" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38287 } }
I20260926 20:42:45.768499 4681 tablet_bootstrap.cc:492] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca: Bootstrap starting.
I20260926 20:42:45.774466 4681 tablet_bootstrap.cc:654] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca: Neither blocks nor log segments found. Creating new log.
I20260926 20:42:45.777107 4374 heartbeater.cc:503] Master 127.23.44.254:34643 was elected leader, sending a full tablet report...
I20260926 20:42:45.778139 4678 ts_tablet_manager.cc:1428] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6: Time spent starting tablet: real 0.059s user 0.036s sys 0.024s
I20260926 20:42:45.786561 4681 log.cc:824] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca: Log is configured to *not* fsync() on all Append() calls
I20260926 20:42:45.791488 4681 tablet_bootstrap.cc:492] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca: No bootstrap required, opened a new log
I20260926 20:42:45.791853 4681 ts_tablet_manager.cc:1397] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.000s
I20260926 20:42:45.806200 4681 raft_consensus.cc:348] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25401a5905074a66bef6fc515a45d289" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40747 } } peers { permanent_uuid: "4d435588d72d4b3caa9a473404d634d6" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41347 } } peers { permanent_uuid: "407ec77a2a2843019749932d38471fca" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38287 } }
I20260926 20:42:45.806732 4681 raft_consensus.cc:374] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:42:45.806847 4681 raft_consensus.cc:729] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 407ec77a2a2843019749932d38471fca, State: Initialized, Role: FOLLOWER
I20260926 20:42:45.807529 4681 consensus_queue.cc:260] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25401a5905074a66bef6fc515a45d289" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40747 } } peers { permanent_uuid: "4d435588d72d4b3caa9a473404d634d6" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41347 } } peers { permanent_uuid: "407ec77a2a2843019749932d38471fca" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38287 } }
I20260926 20:42:45.809518 4681 ts_tablet_manager.cc:1428] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca: Time spent starting tablet: real 0.018s user 0.015s sys 0.005s
W20260926 20:42:45.834296 4654 tablet.cc:2369] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:42:45.861702 4520 tablet.cc:2369] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:42:45.933662 4375 tablet.cc:2369] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:42:45.974603 4682 raft_consensus.cc:482] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:42:45.975006 4682 raft_consensus.cc:504] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25401a5905074a66bef6fc515a45d289" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40747 } } peers { permanent_uuid: "4d435588d72d4b3caa9a473404d634d6" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41347 } } peers { permanent_uuid: "407ec77a2a2843019749932d38471fca" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38287 } }
I20260926 20:42:45.977695 4682 leader_election.cc:288] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25401a5905074a66bef6fc515a45d289 (127.23.44.195:40747), 407ec77a2a2843019749932d38471fca (127.23.44.194:38287)
I20260926 20:42:45.987524 4670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.978386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:60633 (local address 127.23.44.195:40747)
0926 20:42:45.978567 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:45.978573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:45.980484 (+ 1911us) server_negotiation.cc:407] Connection header received
0926 20:42:45.980704 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:45.980711 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:45.980779 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:45.980893 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:42:45.981468 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.982302 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.985249 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.986292 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.986492 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.986584 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:45.987052 (+ 468us) server_negotiation.cc:299] Negotiation successful
0926 20:42:45.987208 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:42:45.987548 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.978379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:60695 (local address 127.23.44.194:38287)
0926 20:42:45.978650 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:45.978660 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:45.979624 (+ 964us) server_negotiation.cc:407] Connection header received
0926 20:42:45.979889 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:45.979895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:45.979999 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:45.980102 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:42:45.980944 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.982036 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.985255 (+ 3219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.986528 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.986688 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.986755 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:45.987175 (+ 420us) server_negotiation.cc:299] Negotiation successful
0926 20:42:45.987313 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:42:45.987826 4693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.977957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:38287 (local address 127.23.44.193:60695)
0926 20:42:45.979460 (+ 1503us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:45.979489 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:45.979719 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:45.980277 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:45.980282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:45.980312 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:42:45.980775 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:45.980788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.982189 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.982195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:45.985021 (+ 2826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:45.985041 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.986765 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.986772 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:45.986969 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.987018 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 20:42:45.987104 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 20:42:45.987249 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1310,"mutex_wait_us":102,"thread_start_us":2101,"threads_started":1}
I20260926 20:42:45.988464 4692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:45.977979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:40747 (local address 127.23.44.193:60633)
0926 20:42:45.980368 (+ 2389us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:45.980385 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:45.980550 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:45.980906 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:45.980911 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:45.980934 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:42:45.981302 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:45.981310 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.982488 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.982495 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:45.985022 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:45.985041 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:45.986538 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:45.986545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:45.986828 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:45.986875 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 20:42:45.987841 (+ 966us) client_negotiation.cc:240] Negotiation successful
0926 20:42:45.988174 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2264,"mutex_wait_us":66,"thread_start_us":1012,"threads_started":1}
I20260926 20:42:45.989164 4474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4daa31fc49d441628bcadf5d34da25e0" candidate_uuid: "4d435588d72d4b3caa9a473404d634d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "407ec77a2a2843019749932d38471fca" is_pre_election: true
I20260926 20:42:45.989483 4608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4daa31fc49d441628bcadf5d34da25e0" candidate_uuid: "4d435588d72d4b3caa9a473404d634d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25401a5905074a66bef6fc515a45d289" is_pre_election: true
I20260926 20:42:45.989833 4474 raft_consensus.cc:2454] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d435588d72d4b3caa9a473404d634d6 in term 0.
I20260926 20:42:45.989918 4608 raft_consensus.cc:2454] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d435588d72d4b3caa9a473404d634d6 in term 0.
I20260926 20:42:45.990679 4263 leader_election.cc:302] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25401a5905074a66bef6fc515a45d289, 4d435588d72d4b3caa9a473404d634d6; no voters:
I20260926 20:42:45.991184 4682 raft_consensus.cc:2781] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:42:45.991338 4682 raft_consensus.cc:482] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:42:45.991478 4682 raft_consensus.cc:3037] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:42:45.993993 4682 raft_consensus.cc:504] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25401a5905074a66bef6fc515a45d289" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40747 } } peers { permanent_uuid: "4d435588d72d4b3caa9a473404d634d6" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41347 } } peers { permanent_uuid: "407ec77a2a2843019749932d38471fca" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38287 } }
I20260926 20:42:45.994670 4682 leader_election.cc:288] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [CANDIDATE]: Term 1 election: Requested vote from peers 25401a5905074a66bef6fc515a45d289 (127.23.44.195:40747), 407ec77a2a2843019749932d38471fca (127.23.44.194:38287)
I20260926 20:42:45.995180 4608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4daa31fc49d441628bcadf5d34da25e0" candidate_uuid: "4d435588d72d4b3caa9a473404d634d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25401a5905074a66bef6fc515a45d289"
I20260926 20:42:45.995294 4474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4daa31fc49d441628bcadf5d34da25e0" candidate_uuid: "4d435588d72d4b3caa9a473404d634d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "407ec77a2a2843019749932d38471fca"
I20260926 20:42:45.995412 4608 raft_consensus.cc:3037] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:42:45.995503 4474 raft_consensus.cc:3037] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:42:45.998656 4608 raft_consensus.cc:2454] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d435588d72d4b3caa9a473404d634d6 in term 1.
I20260926 20:42:45.998656 4474 raft_consensus.cc:2454] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d435588d72d4b3caa9a473404d634d6 in term 1.
I20260926 20:42:45.999131 4264 leader_election.cc:302] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 407ec77a2a2843019749932d38471fca, 4d435588d72d4b3caa9a473404d634d6; no voters:
I20260926 20:42:45.999465 4682 raft_consensus.cc:2781] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:42:46.000375 4682 raft_consensus.cc:686] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [term 1 LEADER]: Becoming Leader. State: Replica: 4d435588d72d4b3caa9a473404d634d6, State: Running, Role: LEADER
I20260926 20:42:46.001372 4682 consensus_queue.cc:237] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25401a5905074a66bef6fc515a45d289" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40747 } } peers { permanent_uuid: "4d435588d72d4b3caa9a473404d634d6" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41347 } } peers { permanent_uuid: "407ec77a2a2843019749932d38471fca" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38287 } }
I20260926 20:42:46.006659 4177 catalog_manager.cc:5495] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d435588d72d4b3caa9a473404d634d6 (127.23.44.193). New cstate: current_term: 1 leader_uuid: "4d435588d72d4b3caa9a473404d634d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25401a5905074a66bef6fc515a45d289" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d435588d72d4b3caa9a473404d634d6" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "407ec77a2a2843019749932d38471fca" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38287 } health_report { overall_health: UNKNOWN } } }
I20260926 20:42:46.045572 4673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:46.038660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38018 (local address 127.23.44.193:41347)
0926 20:42:46.038894 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:46.038900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:46.039239 (+ 339us) server_negotiation.cc:407] Connection header received
0926 20:42:46.039433 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:46.039438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:46.039563 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:46.039731 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:42:46.040143 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:46.041295 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:46.042792 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:46.043379 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:46.043533 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:46.044139 (+ 606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:42:46.044896 (+ 757us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:42:46.045048 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:46.045184 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 20:42:46.045274 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:42:46.054455 4608 raft_consensus.cc:1261] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289 [term 1 FOLLOWER]: Refusing update from remote peer 4d435588d72d4b3caa9a473404d634d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:42:46.054513 4474 raft_consensus.cc:1261] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca [term 1 FOLLOWER]: Refusing update from remote peer 4d435588d72d4b3caa9a473404d634d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:42:46.055519 4694 consensus_queue.cc:1035] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "407ec77a2a2843019749932d38471fca" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 38287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:42:46.055976 4682 consensus_queue.cc:1035] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25401a5905074a66bef6fc515a45d289" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:42:46.077639 4699 mvcc.cc:205] Tried to move back new op lower bound from 7333705179347972096 to 7333705179145748480. Current Snapshot: MvccSnapshot[applied={T|T < 7333705179347972096}]
I20260926 20:42:46.096311 4177 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:42:46.096652 4177 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:42:46.100967 4177 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c942f3fa1b3949388f7a6569e3206161: Sending DeleteTablet for 3 replicas of tablet 4daa31fc49d441628bcadf5d34da25e0
I20260926 20:42:46.102345 4588 tablet_service.cc:1514] Processing DeleteTablet for tablet 4daa31fc49d441628bcadf5d34da25e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:42:46 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:56392
I20260926 20:42:46.103132 4309 tablet_service.cc:1514] Processing DeleteTablet for tablet 4daa31fc49d441628bcadf5d34da25e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:42:46 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:38014
I20260926 20:42:46.104326 4454 tablet_service.cc:1514] Processing DeleteTablet for tablet 4daa31fc49d441628bcadf5d34da25e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:42:46 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:51654
I20260926 20:42:46.105587 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 4214
I20260926 20:42:46.107338 4708 tablet_replica.cc:330] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6: stopping tablet replica
I20260926 20:42:46.108027 4708 raft_consensus.cc:2229] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:42:46.109170 4708 raft_consensus.cc:2258] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:42:46.111739 4708 ts_tablet_manager.cc:1905] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:42:46.112339 4707 tablet_replica.cc:330] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289: stopping tablet replica
I20260926 20:42:46.112838 4707 raft_consensus.cc:2229] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:42:46.113559 4707 raft_consensus.cc:2258] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:42:46.115271 4709 tablet_replica.cc:330] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca: stopping tablet replica
I20260926 20:42:46.116910 4707 ts_tablet_manager.cc:1905] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:42:46.124044 4707 ts_tablet_manager.cc:1918] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:42:46.124256 4707 log.cc:1196] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/4daa31fc49d441628bcadf5d34da25e0
I20260926 20:42:46.126086 4709 raft_consensus.cc:2229] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:42:46.126715 4709 raft_consensus.cc:2258] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:42:46.133148 4708 ts_tablet_manager.cc:1918] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:42:46.133354 4708 log.cc:1196] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/4daa31fc49d441628bcadf5d34da25e0
I20260926 20:42:46.133903 4708 ts_tablet_manager.cc:1939] T 4daa31fc49d441628bcadf5d34da25e0 P 4d435588d72d4b3caa9a473404d634d6: Deleting consensus metadata
I20260926 20:42:46.134485 4707 ts_tablet_manager.cc:1939] T 4daa31fc49d441628bcadf5d34da25e0 P 25401a5905074a66bef6fc515a45d289: Deleting consensus metadata
I20260926 20:42:46.135507 4709 ts_tablet_manager.cc:1905] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:42:46.140796 4162 catalog_manager.cc:4841] TS 25401a5905074a66bef6fc515a45d289 (127.23.44.195:40747): tablet 4daa31fc49d441628bcadf5d34da25e0 (table test-table [id=1d427b2e9cb0460a8313b068ed7115ae]) successfully deleted
I20260926 20:42:46.143076 4709 ts_tablet_manager.cc:1918] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:42:46.143255 4709 log.cc:1196] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/4daa31fc49d441628bcadf5d34da25e0
I20260926 20:42:46.143783 4709 ts_tablet_manager.cc:1939] T 4daa31fc49d441628bcadf5d34da25e0 P 407ec77a2a2843019749932d38471fca: Deleting consensus metadata
I20260926 20:42:46.145690 4163 catalog_manager.cc:4841] TS 407ec77a2a2843019749932d38471fca (127.23.44.194:38287): tablet 4daa31fc49d441628bcadf5d34da25e0 (table test-table [id=1d427b2e9cb0460a8313b068ed7115ae]) successfully deleted
I20260926 20:42:46.147648 4161 catalog_manager.cc:4841] TS 4d435588d72d4b3caa9a473404d634d6 (127.23.44.193:41347): tablet 4daa31fc49d441628bcadf5d34da25e0 (table test-table [id=1d427b2e9cb0460a8313b068ed7115ae]) successfully deleted
I20260926 20:42:46.150048 4673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:46.107039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.23.44.193:41347)
0926 20:42:46.107267 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:46.107276 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:46.115155 (+ 7879us) server_negotiation.cc:407] Connection header received
0926 20:42:46.115656 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:46.115662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:46.115754 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:46.115862 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:46.122695 (+ 6833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:46.123944 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:46.126201 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:46.126954 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:46.127068 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:46.132592 (+ 5524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:46.132610 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:46.132634 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:46.132694 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:46.144516 (+ 11822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:46.145696 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:46.145706 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:46.145710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:46.147516 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:46.148168 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:46.148174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:46.148178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:46.148707 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:46.149024 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:46.149392 (+ 368us) server_negotiation.cc:299] Negotiation successful
0926 20:42:46.149652 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 20:42:46.151175 4369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:46.254241 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 4214
I20260926 20:42:46.273432 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 4379
I20260926 20:42:46.288072 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:46.274222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51656 (local address 127.23.44.194:38287)
0926 20:42:46.274393 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:46.274398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:46.274414 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:42:46.274505 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:46.274509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:46.274568 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:46.274650 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:46.275917 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:46.276998 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:46.278625 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:46.279087 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:46.279210 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:46.282138 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:46.282153 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:46.282171 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:46.282214 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:46.285225 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:46.285980 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:46.285988 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:46.285990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:46.286152 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:46.286658 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:46.286665 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:46.286668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:46.287044 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:46.287292 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:46.287632 (+ 340us) server_negotiation.cc:299] Negotiation successful
0926 20:42:46.287801 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 20:42:46.288825 4514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:46.378556 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 4379
I20260926 20:42:46.395434 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 4523
I20260926 20:42:46.410636 4670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:46.396231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56400 (local address 127.23.44.195:40747)
0926 20:42:46.396417 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:46.396423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:46.396446 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:42:46.396605 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:46.396609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:46.396673 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:46.396777 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:46.398002 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:46.398754 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:46.400194 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:46.400898 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:46.401049 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:46.403726 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:46.403741 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:46.403759 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:46.403818 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:46.407353 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:46.408165 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:46.408173 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:46.408175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:46.408334 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:46.408816 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:46.408822 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:46.408825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:46.409260 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:46.409611 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:46.409913 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 20:42:46.410183 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 20:42:46.411406 4648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:46.502461 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 4523
I20260926 20:42:46.521369 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 4144
I20260926 20:42:46.522897 4207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:46.637912 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 4144
I20260926 20:42:46.668215 23731 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 20:42:47.004856 23731 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 20:42:47.005087 23731 mini_ranger.cc:66] Stopping Ranger...
I20260926 20:42:47.352057 23731 mini_ranger.cc:68] Stopped Ranger
2026-09-26 20:42:47.352 UTC [3195] LOG: received smart shutdown request
2026-09-26 20:42:47.356 UTC [3195] LOG: background worker "logical replication launcher" (PID 3204) exited with exit code 1
2026-09-26 20:42:47.356 UTC [3199] LOG: shutting down
2026-09-26 20:42:47.559 UTC [3195] LOG: database system is shut down
2026-09-26T20:42:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (57575 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:42:47 dist-test-slave-4d5t krb5kdc[4723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:42:47 dist-test-slave-4d5t krb5kdc[4723](info): set up 2 sockets
Sep 26 20:42:47 dist-test-slave-4d5t krb5kdc[4723](info): commencing operation
krb5kdc: starting...
W20260926 20:42:48.861824 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:42:49 dist-test-slave-4d5t krb5kdc[4723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:42:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:42:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:42:49.121970 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:36553
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:36553,127.23.44.253:35941,127.23.44.252:44605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:42:49.467097 4736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:49.467415 4736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:49.467517 4736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:49.475060 4736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:42:49.475206 4736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:49.475257 4736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:49.475310 4736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:42:49.475353 4736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:42:49.485965 4736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:36553,127.23.44.253:35941,127.23.44.252:44605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:36553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:49.487407 4736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:49.488786 4736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:49.501281 4741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:49.501475 4744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:49.501523 4742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:49.503146 4736 server_base.cc:1034] running on GCE node
I20260926 20:42:49.504082 4736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:49.505705 4736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:49.506956 4736 hybrid_clock.cc:648] HybridClock initialized: now 1790455369506915 us; error 65 us; skew 500 ppm
I20260926 20:42:49.507326 4736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:49 dist-test-slave-4d5t krb5kdc[4723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455369, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:49.515067 4736 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:42:49.517536 4736 webserver.cc:460] Webserver started at http://127.23.44.254:43689/ using document root <none> and password file <none>
I20260926 20:42:49.518400 4736 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:49.518569 4736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:42:49.518901 4736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:42:49.522241 4736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "8c5853d71b764c69b802418b814e6f4b"
format_stamp: "Formatted at 2026-09-26 20:42:49 on dist-test-slave-4d5t"
server_key: "aa19f50c2e34b3cbfaf306d7c8223489"
server_key_iv: "a4d267b0baa2e2de8bd987db3f1a2fb7"
server_key_version: "encryptionkey@0"
I20260926 20:42:49.523118 4736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "8c5853d71b764c69b802418b814e6f4b"
format_stamp: "Formatted at 2026-09-26 20:42:49 on dist-test-slave-4d5t"
server_key: "aa19f50c2e34b3cbfaf306d7c8223489"
server_key_iv: "a4d267b0baa2e2de8bd987db3f1a2fb7"
server_key_version: "encryptionkey@0"
I20260926 20:42:49.528977 4736 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260926 20:42:49.533249 4751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:49.535229 4736 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:42:49.535512 4736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "8c5853d71b764c69b802418b814e6f4b"
format_stamp: "Formatted at 2026-09-26 20:42:49 on dist-test-slave-4d5t"
server_key: "aa19f50c2e34b3cbfaf306d7c8223489"
server_key_iv: "a4d267b0baa2e2de8bd987db3f1a2fb7"
server_key_version: "encryptionkey@0"
I20260926 20:42:49.535809 4736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:49.578265 4736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:49.583400 4736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:49.583742 4736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:49.604504 4736 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:36553
I20260926 20:42:49.604544 4802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:36553 every 8 connection(s)
I20260926 20:42:49.605998 4736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:42:49.609500 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 4736
I20260926 20:42:49.609759 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:42:49.610591 4803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:42:49.610702 23731 external_mini_cluster.cc:1428] Setting key 8033df26041e99e1d0d92cfde2081ea3
I20260926 20:42:49.622063 4803 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.44.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.253@KRBTEST.COM" created.
Sep 26 20:42:49 dist-test-slave-4d5t krb5kdc[4723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455369, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:49.654839 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:49.627091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:36553 (local address 127.0.0.1:52686)
0926 20:42:49.628127 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:49.628173 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:49.631052 (+ 2879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:49.632225 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:49.632231 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:49.632928 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:49.633563 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:49.633585 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:49.636087 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:49.636092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:49.637573 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:49.637588 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:49.639227 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:49.639239 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:49.639423 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:49.640182 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:49.640208 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:49.643760 (+ 3552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:49.648362 (+ 4602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:49.648379 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:49.648394 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:49.648961 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:49.649853 (+ 892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:49.649857 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:49.649860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:49.650067 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:49.651611 (+ 1544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:49.651619 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:49.652019 (+ 400us) client_negotiation.cc:769] Sending connection context
0926 20:42:49.652379 (+ 360us) client_negotiation.cc:240] Negotiation successful
0926 20:42:49.653278 (+ 899us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":259,"thread_start_us":444,"threads_started":1}
I20260926 20:42:49.654984 4807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:49.628827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52686 (local address 127.23.44.254:36553)
0926 20:42:49.629724 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:49.629736 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:49.629810 (+ 74us) server_negotiation.cc:407] Connection header received
0926 20:42:49.631576 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:49.631589 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:49.631938 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:49.632198 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:49.633782 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:49.635904 (+ 2122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:49.637736 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:49.638853 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:49.639020 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:49.644087 (+ 5067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:49.644101 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:49.644121 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:49.644174 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:49.648139 (+ 3965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:49.649260 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:49.649276 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:49.649292 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:49.649645 (+ 353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:49.650225 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:49.650234 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:49.650240 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:49.651397 (+ 1157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:49.651723 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:49.652621 (+ 898us) server_negotiation.cc:299] Negotiation successful
0926 20:42:49.653099 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":435,"threads_started":1}
I20260926 20:42:49.658936 4803 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:42:49.660861 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:49.660071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35941 (local address 127.0.0.1:36560)
0926 20:42:49.660477 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:49.660697 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272}
W20260926 20:42:49.661355 4753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
I20260926 20:42:49.663365 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:49.662896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35941 (local address 127.0.0.1:36576)
0926 20:42:49.663125 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:49.663206 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 20:42:49.663587 4803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:35941: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.253.keytab.
Entry for principal kudu/127.23.44.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.253.keytab.
I20260926 20:42:49.675702 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.253:35941
--webserver_interface=127.23.44.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.253.keytab
--principal=kudu/127.23.44.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:36553,127.23.44.253:35941,127.23.44.252:44605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:42:49.712952 4803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } attempt: 1
I20260926 20:42:49.714273 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:49.713712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35941 (local address 127.0.0.1:36580)
0926 20:42:49.714001 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:49.714127 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 20:42:49.715281 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:49.714919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35941 (local address 127.0.0.1:36592)
0926 20:42:49.715087 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:49.715155 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 20:42:49.715477 4803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:35941: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
I20260926 20:42:49.783896 4803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } attempt: 2
I20260926 20:42:49.785444 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:49.784910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35941 (local address 127.0.0.1:36598)
0926 20:42:49.785195 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:49.785291 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 20:42:49.786623 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:49.786224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35941 (local address 127.0.0.1:36600)
0926 20:42:49.786393 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:49.786480 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 20:42:49.786823 4803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:35941: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
I20260926 20:42:49.878108 4803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } attempt: 3
I20260926 20:42:49.879446 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:49.878983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35941 (local address 127.0.0.1:36602)
0926 20:42:49.879201 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:49.879310 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260926 20:42:49.880657 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:49.880231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35941 (local address 127.0.0.1:36614)
0926 20:42:49.880396 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:49.880465 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260926 20:42:49.880824 4803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:35941: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
I20260926 20:42:50.024194 4803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } attempt: 4
I20260926 20:42:50.025513 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.024927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35941 (local address 127.0.0.1:36628)
0926 20:42:50.025278 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.025376 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
I20260926 20:42:50.026660 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.026261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35941 (local address 127.0.0.1:36644)
0926 20:42:50.026420 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.026534 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 20:42:50.026853 4803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:35941: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35941: connect: Connection refused (error 111)
W20260926 20:42:50.044620 4810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:50.044993 4810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:50.045150 4810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:50.053128 4810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:42:50.053306 4810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:50.053407 4810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:50.053517 4810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:42:50.053598 4810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:42:50.066769 4810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:36553,127.23.44.253:35941,127.23.44.252:44605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.253.keytab
--principal=kudu/127.23.44.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.253:35941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.44.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:50.068879 4810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:50.070822 4810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:50.084671 4818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:50.084810 4815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:50.084765 4816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:50.086191 4810 server_base.cc:1034] running on GCE node
I20260926 20:42:50.087224 4810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:50.089006 4810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:50.090391 4810 hybrid_clock.cc:648] HybridClock initialized: now 1790455370090312 us; error 38 us; skew 500 ppm
I20260926 20:42:50.090997 4810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:50 dist-test-slave-4d5t krb5kdc[4723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455370, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:50.095322 4810 init.cc:377] Logged in from keytab as kudu/127.23.44.253@KRBTEST.COM (short username kudu)
I20260926 20:42:50.097389 4810 webserver.cc:460] Webserver started at http://127.23.44.253:44467/ using document root <none> and password file <none>
I20260926 20:42:50.098265 4810 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:50.098428 4810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:42:50.098798 4810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:42:50.102560 4810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/data/instance:
uuid: "89aa87732e894f3795049a3c0e818cd5"
format_stamp: "Formatted at 2026-09-26 20:42:50 on dist-test-slave-4d5t"
server_key: "79ff75398fd8bebb68bf9e99df201d90"
server_key_iv: "543c80f924737f03df8b59d133411d41"
server_key_version: "encryptionkey@0"
I20260926 20:42:50.103518 4810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/wal/instance:
uuid: "89aa87732e894f3795049a3c0e818cd5"
format_stamp: "Formatted at 2026-09-26 20:42:50 on dist-test-slave-4d5t"
server_key: "79ff75398fd8bebb68bf9e99df201d90"
server_key_iv: "543c80f924737f03df8b59d133411d41"
server_key_version: "encryptionkey@0"
I20260926 20:42:50.109736 4810 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 20:42:50.113476 4825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:50.115378 4810 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:42:50.115558 4810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/wal
uuid: "89aa87732e894f3795049a3c0e818cd5"
format_stamp: "Formatted at 2026-09-26 20:42:50 on dist-test-slave-4d5t"
server_key: "79ff75398fd8bebb68bf9e99df201d90"
server_key_iv: "543c80f924737f03df8b59d133411d41"
server_key_version: "encryptionkey@0"
I20260926 20:42:50.115772 4810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:50.139364 4810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:50.144176 4810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:50.144433 4810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:50.165011 4810 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.253:35941
I20260926 20:42:50.165098 4876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.253:35941 every 8 connection(s)
I20260926 20:42:50.166427 4810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/data/info.pb
I20260926 20:42:50.170550 4877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:42:50.174319 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 4810
I20260926 20:42:50.174530 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-1/wal/instance
I20260926 20:42:50.175017 23731 external_mini_cluster.cc:1428] Setting key 53d55f13a5f294914295b4b3f50a37ba
I20260926 20:42:50.181833 4877 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.44.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.252@KRBTEST.COM" created.
Sep 26 20:42:50 dist-test-slave-4d5t krb5kdc[4723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455370, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.253@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:50.209738 4880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.185520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52692 (local address 127.23.44.254:36553)
0926 20:42:50.186102 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.186109 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.187265 (+ 1156us) server_negotiation.cc:407] Connection header received
0926 20:42:50.189494 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.189503 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.189611 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.189759 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.191571 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.192817 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.195343 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.196169 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.196327 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.203039 (+ 6712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.203045 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.203049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.203105 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.206147 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.207099 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.207106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.207110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.207246 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.207718 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.207729 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.207731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.208043 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.208220 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.209065 (+ 845us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.209310 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":306,"threads_started":1}
I20260926 20:42:50.211223 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.186202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:36553 (local address 127.0.0.1:52692)
0926 20:42:50.187073 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.187127 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:50.189142 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:50.190077 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:50.190086 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:50.190678 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:50.191332 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.191351 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.193161 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.193168 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.195117 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.195141 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.196460 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.196466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.196812 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.197692 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:50.197730 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:50.202707 (+ 4977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:50.206374 (+ 3667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.206393 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.206412 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.206944 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.207369 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.207374 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.207379 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.207583 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.208190 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:50.208199 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:50.208667 (+ 468us) client_negotiation.cc:769] Sending connection context
0926 20:42:50.208979 (+ 312us) client_negotiation.cc:240] Negotiation successful
0926 20:42:50.209562 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":373,"threads_started":1}
I20260926 20:42:50.213651 4877 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:42:50 dist-test-slave-4d5t krb5kdc[4723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455370, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.253@KRBTEST.COM for kudu/127.23.44.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.252.keytab.
Entry for principal kudu/127.23.44.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.252.keytab.
I20260926 20:42:50.233435 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.252:44605
--webserver_interface=127.23.44.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.252.keytab
--principal=kudu/127.23.44.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:36553,127.23.44.253:35941,127.23.44.252:44605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:42:50.233536 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.214788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35941 (local address 127.0.0.1:36652)
0926 20:42:50.215066 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.215094 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:50.215232 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:50.216498 (+ 1266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:50.216502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:50.217093 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:50.217248 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.217257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.218965 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.218972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.220087 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.220100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.221129 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.221135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.221403 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.222107 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:50.222137 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:50.226286 (+ 4149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:50.230009 (+ 3723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.230015 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.230018 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.230685 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.231223 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.231227 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.231230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.231489 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.232264 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:50.232270 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:50.232427 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 20:42:50.232669 (+ 242us) client_negotiation.cc:240] Negotiation successful
0926 20:42:50.233051 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":6}
I20260926 20:42:50.233587 4883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.215152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36652 (local address 127.23.44.253:35941)
0926 20:42:50.215956 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.215963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.215995 (+ 32us) server_negotiation.cc:407] Connection header received
0926 20:42:50.216048 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.216055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.216312 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.216513 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.217416 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.218750 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.220272 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.220954 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.221125 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.226544 (+ 5419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.226558 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.226583 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.226619 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.229786 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.230931 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.230939 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.230942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.231081 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.231611 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.231614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.231616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.232080 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.232301 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.232844 (+ 543us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.233182 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":126,"server-negotiator.queue_time_us":636,"thread_start_us":470,"threads_started":1}
I20260926 20:42:50.236410 4877 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:42:50.238055 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.237320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40356)
0926 20:42:50.237642 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.237810 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260926 20:42:50.238528 4828 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
I20260926 20:42:50.241302 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.240837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40364)
0926 20:42:50.241043 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.241155 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 20:42:50.241691 4877 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:44605: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
I20260926 20:42:50.291069 4877 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } attempt: 1
I20260926 20:42:50.292446 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.291868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40366)
0926 20:42:50.292119 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.292251 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 20:42:50.294004 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.293524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40372)
0926 20:42:50.293752 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.293859 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260926 20:42:50.294349 4877 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:44605: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
I20260926 20:42:50.326124 4803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } attempt: 5
Sep 26 20:42:50 dist-test-slave-4d5t krb5kdc[4723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455369, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for kudu/127.23.44.253@KRBTEST.COM
I20260926 20:42:50.342199 4883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.326981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.23.44.253:35941)
0926 20:42:50.327167 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.327172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.327254 (+ 82us) server_negotiation.cc:407] Connection header received
0926 20:42:50.327586 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.327593 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.327704 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.327855 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.328991 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.329921 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.331798 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.332291 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.332399 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.336229 (+ 3830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.336236 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.336239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.336275 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.339253 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.340046 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.340052 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.340056 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.340223 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.340653 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.340659 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.340662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.341028 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.341205 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.341649 (+ 444us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.341872 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 20:42:50.342236 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.326870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35941 (local address 127.0.0.1:36666)
0926 20:42:50.327136 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.327163 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:50.327325 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:50.327936 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:50.327941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:50.328460 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:50.328775 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.328788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.330132 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.330138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.331580 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.331595 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.332474 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.332481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.332736 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.333444 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:50.333469 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:50.335993 (+ 2524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:50.339413 (+ 3420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.339421 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.339426 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.339922 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.340345 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.340350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.340354 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.340492 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.341162 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:50.341167 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:50.341367 (+ 200us) client_negotiation.cc:769] Sending connection context
0926 20:42:50.341524 (+ 157us) client_negotiation.cc:240] Negotiation successful
0926 20:42:50.341863 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260926 20:42:50.343194 4803 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:42:50.344399 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.344007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40386)
0926 20:42:50.344185 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.344264 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260926 20:42:50.345770 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.345398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40392)
0926 20:42:50.345568 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.345652 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260926 20:42:50.346010 4803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:44605: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
I20260926 20:42:50.362782 4877 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } attempt: 2
I20260926 20:42:50.364732 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.364006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40400)
0926 20:42:50.364297 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.364447 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260926 20:42:50.366139 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.365617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40402)
0926 20:42:50.365809 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.365920 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 20:42:50.366487 4877 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:44605: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
I20260926 20:42:50.397265 4803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } attempt: 1
I20260926 20:42:50.398584 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.397987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40412)
0926 20:42:50.398249 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.398369 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260926 20:42:50.399932 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.399521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40424)
0926 20:42:50.399682 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.399772 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 20:42:50.400314 4803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:44605: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
I20260926 20:42:50.457794 4877 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } attempt: 3
I20260926 20:42:50.459075 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.458527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40428)
0926 20:42:50.458765 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.458878 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 20:42:50.461299 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.460291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40442)
0926 20:42:50.460755 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.460960 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
W20260926 20:42:50.461720 4877 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:44605: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
I20260926 20:42:50.468638 4803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } attempt: 2
I20260926 20:42:50.470865 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.469865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40458)
0926 20:42:50.470236 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.470429 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260926 20:42:50.472132 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.471672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40470)
0926 20:42:50.471850 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.471968 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260926 20:42:50.472375 4803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:44605: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
I20260926 20:42:50.578999 4803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } attempt: 3
I20260926 20:42:50.580624 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.579932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40478)
0926 20:42:50.580236 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.580369 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260926 20:42:50.582046 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.581551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40482)
0926 20:42:50.581808 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.581910 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260926 20:42:50.582320 4803 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:44605: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
W20260926 20:42:50.601503 4885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:50.601856 4885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:50.602028 4885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 20:42:50.604974 4877 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } attempt: 4
I20260926 20:42:50.606257 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.605687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40486)
0926 20:42:50.605944 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.606063 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 20:42:50.607370 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.606983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40492)
0926 20:42:50.607147 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.607236 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 20:42:50.607621 4877 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:44605: Network error: Client connection negotiation failed: client connection to 127.23.44.252:44605: connect: Connection refused (error 111)
W20260926 20:42:50.610538 4885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:42:50.610643 4885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:50.610772 4885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:50.610872 4885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:42:50.610946 4885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:42:50.622328 4885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:36553,127.23.44.253:35941,127.23.44.252:44605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.252.keytab
--principal=kudu/127.23.44.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.252:44605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.44.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:50.623798 4885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:50.625324 4885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:50.638293 4893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:50.638607 4890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:50.638346 4891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:50.639645 4885 server_base.cc:1034] running on GCE node
I20260926 20:42:50.640704 4885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:50.642328 4885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:50.643580 4885 hybrid_clock.cc:648] HybridClock initialized: now 1790455370643527 us; error 57 us; skew 500 ppm
I20260926 20:42:50.643981 4885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:50 dist-test-slave-4d5t krb5kdc[4723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455370, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:50.647688 4885 init.cc:377] Logged in from keytab as kudu/127.23.44.252@KRBTEST.COM (short username kudu)
I20260926 20:42:50.650278 4885 webserver.cc:460] Webserver started at http://127.23.44.252:37351/ using document root <none> and password file <none>
I20260926 20:42:50.651203 4885 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:50.651376 4885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:42:50.651865 4885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:42:50.655216 4885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/data/instance:
uuid: "c6664b7ca47d48e5818a4a9e33810b13"
format_stamp: "Formatted at 2026-09-26 20:42:50 on dist-test-slave-4d5t"
server_key: "9a2b12f7c39ad2be7462fbd9a2aa60db"
server_key_iv: "2ee1627dd93a4b16ee708bbd37339c3c"
server_key_version: "encryptionkey@0"
I20260926 20:42:50.656133 4885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/wal/instance:
uuid: "c6664b7ca47d48e5818a4a9e33810b13"
format_stamp: "Formatted at 2026-09-26 20:42:50 on dist-test-slave-4d5t"
server_key: "9a2b12f7c39ad2be7462fbd9a2aa60db"
server_key_iv: "2ee1627dd93a4b16ee708bbd37339c3c"
server_key_version: "encryptionkey@0"
I20260926 20:42:50.661651 4885 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 20:42:50.665601 4900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:50.667042 4885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:42:50.667325 4885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/wal
uuid: "c6664b7ca47d48e5818a4a9e33810b13"
format_stamp: "Formatted at 2026-09-26 20:42:50 on dist-test-slave-4d5t"
server_key: "9a2b12f7c39ad2be7462fbd9a2aa60db"
server_key_iv: "2ee1627dd93a4b16ee708bbd37339c3c"
server_key_version: "encryptionkey@0"
I20260926 20:42:50.667546 4885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:50.690218 4885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:50.696725 4885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:50.697109 4885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:50.718796 4885 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.252:44605
I20260926 20:42:50.718945 4952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.252:44605 every 8 connection(s)
I20260926 20:42:50.720400 4885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/data/info.pb
I20260926 20:42:50.724948 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:42:50.730598 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 4885
I20260926 20:42:50.730856 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790455151607908-23731-0/minicluster-data/master-2/wal/instance
I20260926 20:42:50.731412 23731 external_mini_cluster.cc:1428] Setting key b00138dde9b0f8945e48d1f388804af1
I20260926 20:42:50.732618 4953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:42:50 dist-test-slave-4d5t krb5kdc[4723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455370, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.252@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
Sep 26 20:42:50 dist-test-slave-4d5t krb5kdc[4723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.252@KRBTEST.COM
I20260926 20:42:50.758285 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.735588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52698 (local address 127.23.44.254:36553)
0926 20:42:50.736231 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.736238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.736803 (+ 565us) server_negotiation.cc:407] Connection header received
0926 20:42:50.737505 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.737511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.737602 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.737745 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.739721 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.741438 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.743989 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.744802 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.744951 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.749545 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.749551 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.749554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.749591 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.753431 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.754948 (+ 1517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.754953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.754955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.755116 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.755965 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.755970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.755974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.756427 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.756545 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.757562 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.757816 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":293,"threads_started":1}
I20260926 20:42:50.759404 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.733885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40494 (local address 127.23.44.252:44605)
0926 20:42:50.734757 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.734768 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.734810 (+ 42us) server_negotiation.cc:407] Connection header received
0926 20:42:50.737232 (+ 2422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.737256 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.737583 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.737806 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.739726 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.741927 (+ 2201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.744030 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.745106 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.745238 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.750505 (+ 5267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.750524 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.750545 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.750603 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.754588 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.756000 (+ 1412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.756010 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.756016 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.756188 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.756855 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.756859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.756861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.757311 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.757679 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.758071 (+ 392us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.758327 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":396,"threads_started":1}
I20260926 20:42:50.759523 4958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.735598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:36553 (local address 127.0.0.1:52698)
0926 20:42:50.736637 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.736692 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:50.737246 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:50.737839 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:50.737845 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:50.738778 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:50.739412 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.739452 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.741625 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.741630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.743761 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.743781 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.745045 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.745050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.745244 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.746118 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:50.746154 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:50.749298 (+ 3144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:50.753605 (+ 4307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.753614 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.753626 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.754768 (+ 1142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.755324 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.755330 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.755336 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.755771 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.756675 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:50.756685 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:50.757144 (+ 459us) client_negotiation.cc:769] Sending connection context
0926 20:42:50.757412 (+ 268us) client_negotiation.cc:240] Negotiation successful
0926 20:42:50.758043 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"spinlock_wait_cycles":26496,"thread_start_us":566,"threads_started":1}
I20260926 20:42:50.759554 4803 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } attempt: 4
I20260926 20:42:50.761401 4953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:42:50 dist-test-slave-4d5t krb5kdc[4723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455369, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for kudu/127.23.44.252@KRBTEST.COM
Sep 26 20:42:50 dist-test-slave-4d5t krb5kdc[4723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455370, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.252@KRBTEST.COM for kudu/127.23.44.253@KRBTEST.COM
Sep 26 20:42:50 dist-test-slave-4d5t krb5kdc[4723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:50.776857 4806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.760110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40496)
0926 20:42:50.760294 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.760315 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:50.760427 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:50.761347 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:50.761353 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:50.761825 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:50.762032 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.762044 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.763373 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.763379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.764870 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.764884 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.765755 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.765763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.765989 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.766742 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:50.766766 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:50.769392 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:50.773876 (+ 4484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.773882 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.773887 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.774386 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.774801 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.774809 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.774813 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.774964 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.775618 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:50.775623 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:50.775823 (+ 200us) client_negotiation.cc:769] Sending connection context
0926 20:42:50.775987 (+ 164us) client_negotiation.cc:240] Negotiation successful
0926 20:42:50.776350 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 20:42:50.778038 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.760846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40496 (local address 127.23.44.252:44605)
0926 20:42:50.761003 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.761008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.761024 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:42:50.761066 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.761084 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.761183 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.761291 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.762215 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.763255 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.765034 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.765596 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.765726 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.769599 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.769606 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.769613 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.769658 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.773721 (+ 4063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.774524 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.774531 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.774535 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.774679 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.775084 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.775091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.775095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.775486 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.775646 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.777368 (+ 1722us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.777664 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 20:42:50.778563 4883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.762333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36678 (local address 127.23.44.253:35941)
0926 20:42:50.762548 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.762556 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.762716 (+ 160us) server_negotiation.cc:407] Connection header received
0926 20:42:50.762907 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.762912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.763028 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.763137 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.764251 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.765346 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.767011 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.767727 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.767862 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.771658 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.771663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.771667 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.771701 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.775269 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.776004 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.776012 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.776015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.776168 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.776622 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.776626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.776628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.776886 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.777081 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.777582 (+ 501us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.777861 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 20:42:50.778813 4958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.762337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35941 (local address 127.0.0.1:36678)
0926 20:42:50.762614 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.762635 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:50.762768 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:50.763317 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:50.763325 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:50.763826 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:50.764079 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.764091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.765523 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.765527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.766844 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.766860 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.767911 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.767919 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.768160 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.768815 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:50.768839 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:50.771446 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:50.775400 (+ 3954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.775406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.775410 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.775878 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.776289 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.776295 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.776299 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.776460 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.777144 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:50.777149 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:50.777311 (+ 162us) client_negotiation.cc:769] Sending connection context
0926 20:42:50.777470 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 20:42:50.777857 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":73}
I20260926 20:42:50.779991 4953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:42:50.782385 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.763171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52710 (local address 127.23.44.254:36553)
0926 20:42:50.763634 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.763638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.763651 (+ 13us) server_negotiation.cc:407] Connection header received
0926 20:42:50.763685 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.763688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.763757 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.763865 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.765437 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.766457 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.768315 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.769086 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.769241 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.775422 (+ 6181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.775429 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.775433 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.775475 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.778869 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.779805 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.779810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.779814 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.779948 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.780420 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.780428 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.780434 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.780816 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.781121 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.781725 (+ 604us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.782011 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348}
I20260926 20:42:50.784426 4803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b: Bootstrap starting.
I20260926 20:42:50.788587 4803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b: Neither blocks nor log segments found. Creating new log.
Sep 26 20:42:50 dist-test-slave-4d5t krb5kdc[4723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455370, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.252@KRBTEST.COM for kudu/127.23.44.252@KRBTEST.COM
I20260926 20:42:50.790050 4803 log.cc:824] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b: Log is configured to *not* fsync() on all Append() calls
I20260926 20:42:50.793658 4803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b: No bootstrap required, opened a new log
Sep 26 20:42:50 dist-test-slave-4d5t krb5kdc[4723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.253@KRBTEST.COM
I20260926 20:42:50.797137 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.780869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40500 (local address 127.23.44.252:44605)
0926 20:42:50.781044 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.781051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.781239 (+ 188us) server_negotiation.cc:407] Connection header received
0926 20:42:50.781426 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.781432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.781538 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.781633 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.782668 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.783826 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.785512 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.785993 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.786123 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.790192 (+ 4069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.790198 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.790202 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.790239 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.793768 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.794708 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.794713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.794717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.794866 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.795323 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.795332 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.795335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.795657 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.795826 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.796315 (+ 489us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.796686 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":61}
I20260926 20:42:50.797555 4958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.780869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40500)
0926 20:42:50.781146 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.781170 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:50.781297 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:50.781794 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:50.781799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:50.782289 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:50.782506 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.782522 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.783982 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.783989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.785381 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.785393 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.786144 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.786149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.786340 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.786965 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:50.787027 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:50.789964 (+ 2937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:50.793928 (+ 3964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.793936 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.793943 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.794559 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.795006 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.795013 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.795018 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.795201 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.795882 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:50.795887 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:50.796039 (+ 152us) client_negotiation.cc:769] Sending connection context
0926 20:42:50.796211 (+ 172us) client_negotiation.cc:240] Negotiation successful
0926 20:42:50.796824 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":254}
I20260926 20:42:50.802986 4953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13: Bootstrap starting.
I20260926 20:42:50.804543 4883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.784399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.23.44.253:35941)
0926 20:42:50.784640 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.784647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.784799 (+ 152us) server_negotiation.cc:407] Connection header received
0926 20:42:50.784965 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.784971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.785045 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.785151 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.787091 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.788401 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.790213 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.790943 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.791074 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.796992 (+ 5918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.796999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.797005 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.797052 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.800679 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.801672 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.801679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.801684 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.801847 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.802286 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.802290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.802295 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.802646 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.802935 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.803917 (+ 982us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.804171 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 20:42:50.804389 4803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } }
I20260926 20:42:50.804826 4803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:42:50.804941 4803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c5853d71b764c69b802418b814e6f4b, State: Initialized, Role: FOLLOWER
I20260926 20:42:50.805663 4803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } }
I20260926 20:42:50.807062 4960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } } }
I20260926 20:42:50.807421 4960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:42:50.807441 4953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13: Neither blocks nor log segments found. Creating new log.
I20260926 20:42:50.808672 4953 log.cc:824] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13: Log is configured to *not* fsync() on all Append() calls
I20260926 20:42:50.809386 4803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:42:50.810362 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:42:50.812094 4953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13: No bootstrap required, opened a new log
I20260926 20:42:50.826159 4953 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } }
I20260926 20:42:50.826730 4953 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:42:50.826881 4953 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6664b7ca47d48e5818a4a9e33810b13, State: Initialized, Role: FOLLOWER
I20260926 20:42:50.831583 4953 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } }
W20260926 20:42:50.837535 4757 tablet.cc:2369] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:42:50.843607 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.820232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52726 (local address 127.23.44.254:36553)
0926 20:42:50.820521 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.820530 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.821289 (+ 759us) server_negotiation.cc:407] Connection header received
0926 20:42:50.821624 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.821631 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.821724 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.821807 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.824599 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.825800 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.828525 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.829367 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.829494 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.833944 (+ 4450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.833951 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.833957 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.834014 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.838487 (+ 4473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.840092 (+ 1605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.840100 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.840105 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.840311 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.841124 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.841131 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.841135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.841620 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.841990 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.842750 (+ 760us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.843080 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260926 20:42:50.843873 4978 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } } }
I20260926 20:42:50.844372 4978 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:42:50.846607 4953 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:42:50.852439 4883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.822255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36686 (local address 127.23.44.253:35941)
0926 20:42:50.822425 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.822431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.823052 (+ 621us) server_negotiation.cc:407] Connection header received
0926 20:42:50.824041 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.824051 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.824158 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.824245 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.828176 (+ 3931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.829680 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.834662 (+ 4982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.835841 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.836031 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.841815 (+ 5784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.841827 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.841834 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.841903 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.847473 (+ 5570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.848709 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.848720 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.848728 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.848973 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.849598 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.849608 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.849614 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.850233 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.850600 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.851416 (+ 816us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.851793 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 20:42:50.860759 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.821817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40510 (local address 127.23.44.252:44605)
0926 20:42:50.822017 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.822024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.827899 (+ 5875us) server_negotiation.cc:407] Connection header received
0926 20:42:50.828203 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.828211 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.828306 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.828434 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.832350 (+ 3916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.833602 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.836009 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.836880 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.837031 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.849018 (+ 11987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.849027 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.849036 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.849089 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.854417 (+ 5328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.855372 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.855383 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.855390 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.855597 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.856235 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.856244 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.856250 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.857175 (+ 925us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.859392 (+ 2217us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.859618 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.860012 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260926 20:42:50.871775 4982 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:42:50.871942 4982 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:42:50.863488 4736 master_runner.cc:186] Error getting master registration for 127.23.44.253:35941: OK, instance_id { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" instance_seqno: 1790455370158989 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 20:42:50.876204 4736 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:42:50.877974 4978 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:42:50.878288 4978 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } }
I20260926 20:42:50.880365 4978 leader_election.cc:288] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c5853d71b764c69b802418b814e6f4b (127.23.44.254:36553), 89aa87732e894f3795049a3c0e818cd5 (127.23.44.253:35941)
I20260926 20:42:50.882088 4778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6664b7ca47d48e5818a4a9e33810b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c5853d71b764c69b802418b814e6f4b" is_pre_election: true
I20260926 20:42:50.882742 4778 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6664b7ca47d48e5818a4a9e33810b13 in term 0.
I20260926 20:42:50.883400 4904 leader_election.cc:302] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c5853d71b764c69b802418b814e6f4b, c6664b7ca47d48e5818a4a9e33810b13; no voters:
I20260926 20:42:50.883872 4978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:42:50.884054 4978 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:42:50.884179 4978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:42:50.889549 4978 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } }
I20260926 20:42:50.891330 4978 leader_election.cc:288] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [CANDIDATE]: Term 1 election: Requested vote from peers 8c5853d71b764c69b802418b814e6f4b (127.23.44.254:36553), 89aa87732e894f3795049a3c0e818cd5 (127.23.44.253:35941)
I20260926 20:42:50.891566 4777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6664b7ca47d48e5818a4a9e33810b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c5853d71b764c69b802418b814e6f4b"
I20260926 20:42:50.891891 4777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:42:50.895555 4777 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6664b7ca47d48e5818a4a9e33810b13 in term 1.
W20260926 20:42:50.896836 4992 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:42:50.896993 4992 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:42:50.897517 4904 leader_election.cc:302] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c5853d71b764c69b802418b814e6f4b, c6664b7ca47d48e5818a4a9e33810b13; no voters:
I20260926 20:42:50.897949 4978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:42:50.907441 4877 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } attempt: 5
I20260926 20:42:50.885696 4885 master_runner.cc:186] Error getting master registration for 127.23.44.253:35941: OK, instance_id { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" instance_seqno: 1790455370158989 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 20:42:50.910363 4885 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:42:50.916033 4978 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [term 1 LEADER]: Becoming Leader. State: Replica: c6664b7ca47d48e5818a4a9e33810b13, State: Running, Role: LEADER
I20260926 20:42:50.916723 4978 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } }
Sep 26 20:42:50 dist-test-slave-4d5t krb5kdc[4723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455370, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.253@KRBTEST.COM for kudu/127.23.44.252@KRBTEST.COM
I20260926 20:42:50.928856 4994 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6664b7ca47d48e5818a4a9e33810b13. Latest consensus state: current_term: 1 leader_uuid: "c6664b7ca47d48e5818a4a9e33810b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } } }
I20260926 20:42:50.929378 4994 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [sys.catalog]: This master's current role is: LEADER
I20260926 20:42:50.930135 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.908161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:44605 (local address 127.0.0.1:40526)
0926 20:42:50.908428 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:50.908449 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:50.908797 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:50.909350 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:50.909356 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:50.909929 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:50.910139 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.910152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.911741 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.911749 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.913812 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:50.913836 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.915162 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.915169 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:50.915477 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.916332 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:50.916354 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:50.920458 (+ 4104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:50.926231 (+ 5773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.926237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.926242 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.927291 (+ 1049us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.927915 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:50.927920 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:50.927924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:50.928107 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:50.928707 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:50.928714 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:50.928895 (+ 181us) client_negotiation.cc:769] Sending connection context
0926 20:42:50.929049 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 20:42:50.929759 (+ 710us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260926 20:42:50.930770 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:50.908207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40526 (local address 127.23.44.252:44605)
0926 20:42:50.908716 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:50.908722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:50.908744 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:42:50.908954 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:50.908962 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:50.909044 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:50.909133 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:50.910320 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.911554 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.914010 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:50.914777 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:50.914915 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:50.920920 (+ 6005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:50.920928 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:50.920934 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:50.921004 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:50.925953 (+ 4949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.927510 (+ 1557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.927517 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.927520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.927708 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:50.928228 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:50.928233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:50.928236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:50.928506 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:50.928854 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:50.929352 (+ 498us) server_negotiation.cc:299] Negotiation successful
0926 20:42:50.929894 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383}
I20260926 20:42:50.935963 4877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5: Bootstrap starting.
I20260926 20:42:50.941116 4877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5: Neither blocks nor log segments found. Creating new log.
I20260926 20:42:50.942895 4877 log.cc:824] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5: Log is configured to *not* fsync() on all Append() calls
I20260926 20:42:50.943977 4996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:42:50.949630 4996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260926 20:42:50.949913 4906 tablet.cc:2369] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:42:50.958192 4877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5: No bootstrap required, opened a new log
I20260926 20:42:50.970211 4877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } }
I20260926 20:42:50.970764 4877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:42:50.970913 4877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89aa87732e894f3795049a3c0e818cd5, State: Initialized, Role: FOLLOWER
I20260926 20:42:50.971675 4877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } }
I20260926 20:42:50.986665 4777 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [term 1 FOLLOWER]: Refusing update from remote peer c6664b7ca47d48e5818a4a9e33810b13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:42:50.987707 4994 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:42:50.990653 4877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:42:50.993230 4851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6664b7ca47d48e5818a4a9e33810b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89aa87732e894f3795049a3c0e818cd5"
I20260926 20:42:50.994273 4852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6664b7ca47d48e5818a4a9e33810b13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89aa87732e894f3795049a3c0e818cd5" is_pre_election: true
I20260926 20:42:50.996869 4851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:42:51.001951 4851 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6664b7ca47d48e5818a4a9e33810b13 in term 1.
I20260926 20:42:51.006026 4850 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [term 1 FOLLOWER]: Refusing update from remote peer c6664b7ca47d48e5818a4a9e33810b13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:42:51.005585 4999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } } }
I20260926 20:42:51.006223 4999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:42:51.007602 4994 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:42:51.038009 4999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6664b7ca47d48e5818a4a9e33810b13. Latest consensus state: current_term: 1 leader_uuid: "c6664b7ca47d48e5818a4a9e33810b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } } }
I20260926 20:42:51.038368 4999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:42:51.048635 4978 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6664b7ca47d48e5818a4a9e33810b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } } }
I20260926 20:42:51.049081 4978 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [sys.catalog]: This master's current role is: LEADER
I20260926 20:42:51.049419 5003 mvcc.cc:205] Tried to move back new op lower bound from 7333705199514968064 to 7333705199279988736. Current Snapshot: MvccSnapshot[applied={T|T < 7333705199514968064}]
I20260926 20:42:51.053874 4996 catalog_manager.cc:1347] Generated new cluster ID: 1d97e0c254c345ea8eae49025b072414
I20260926 20:42:51.054028 4996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:42:51.055917 4999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6664b7ca47d48e5818a4a9e33810b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } } }
I20260926 20:42:51.056191 4999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:42:51.076613 4960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6664b7ca47d48e5818a4a9e33810b13. Latest consensus state: current_term: 1 leader_uuid: "c6664b7ca47d48e5818a4a9e33810b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } } }
I20260926 20:42:51.077013 4960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:42:51.080031 4996 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:42:51.081158 5017 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 20:42:51.082049 4996 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:42:51.085337 4994 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6664b7ca47d48e5818a4a9e33810b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } } }
I20260926 20:42:51.085785 4994 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13 [sys.catalog]: This master's current role is: LEADER
I20260926 20:42:51.099623 4960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6664b7ca47d48e5818a4a9e33810b13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c5853d71b764c69b802418b814e6f4b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 36553 } } peers { permanent_uuid: "89aa87732e894f3795049a3c0e818cd5" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35941 } } peers { permanent_uuid: "c6664b7ca47d48e5818a4a9e33810b13" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 44605 } } }
I20260926 20:42:51.099870 4960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:42:51.100598 5001 mvcc.cc:205] Tried to move back new op lower bound from 7333705199514968064 to 7333705199279988736. Current Snapshot: MvccSnapshot[applied={T|T < 7333705199514968064 or (T in {7333705199514968064})}]
I20260926 20:42:51.111323 4996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6664b7ca47d48e5818a4a9e33810b13: Generated new TSK 0
I20260926 20:42:51.112545 4996 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:42:51.873992 4982 catalog_manager.cc:1259] Loaded cluster ID: 1d97e0c254c345ea8eae49025b072414
I20260926 20:42:51.874100 4982 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b: loading cluster ID for follower catalog manager: success
I20260926 20:42:51.877032 4982 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b: acquiring CA information for follower catalog manager: success
I20260926 20:42:51.878943 4982 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8c5853d71b764c69b802418b814e6f4b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 20:42:52.086135 5017 catalog_manager.cc:1259] Loaded cluster ID: 1d97e0c254c345ea8eae49025b072414
I20260926 20:42:52.086249 5017 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5: loading cluster ID for follower catalog manager: success
I20260926 20:42:52.089099 5017 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5: acquiring CA information for follower catalog manager: success
I20260926 20:42:52.091169 5017 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 89aa87732e894f3795049a3c0e818cd5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 20:42:52.205030 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 4736
I20260926 20:42:52.206682 4798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:52.321224 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 4736
I20260926 20:42:52.343425 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 4810
I20260926 20:42:52.344694 4872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:52.460836 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 4810
I20260926 20:42:52.481539 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 4885
I20260926 20:42:52.482685 4948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:52.595319 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 4885
2026-09-26T20:42:52Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4900 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:42:52 dist-test-slave-4d5t krb5kdc[5032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:42:52 dist-test-slave-4d5t krb5kdc[5032](info): set up 2 sockets
Sep 26 20:42:52 dist-test-slave-4d5t krb5kdc[5032](info): commencing operation
krb5kdc: starting...
W20260926 20:42:53.761406 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:42:53 dist-test-slave-4d5t krb5kdc[5032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:42:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:42:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:42:54.025103 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:41387
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:41387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:42:54.385392 5045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:54.385990 5045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:54.386243 5045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:54.395620 5045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:42:54.395812 5045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:54.395931 5045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:54.396052 5045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:42:54.396137 5045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:42:54.407198 5045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:41387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:41387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:54.408746 5045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:54.410804 5045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:54.423724 5050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:54.423627 5051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:54.423695 5053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:54.424919 5045 server_base.cc:1034] running on GCE node
I20260926 20:42:54.425911 5045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:54.427584 5045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:54.428869 5045 hybrid_clock.cc:648] HybridClock initialized: now 1790455374428805 us; error 59 us; skew 500 ppm
I20260926 20:42:54.429280 5045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:54 dist-test-slave-4d5t krb5kdc[5032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455374, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:54.432448 5045 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:42:54.434146 5045 webserver.cc:460] Webserver started at http://127.23.44.254:39323/ using document root <none> and password file <none>
I20260926 20:42:54.434796 5045 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:54.434898 5045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:42:54.435170 5045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:42:54.438259 5045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "d048786285b14ee19d80cc86cfe8361e"
format_stamp: "Formatted at 2026-09-26 20:42:54 on dist-test-slave-4d5t"
server_key: "419e238deb335ea52a3b03a1d72bddbb"
server_key_iv: "f487a4449058ec53d4e0c98e27c4b5d0"
server_key_version: "encryptionkey@0"
I20260926 20:42:54.439031 5045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "d048786285b14ee19d80cc86cfe8361e"
format_stamp: "Formatted at 2026-09-26 20:42:54 on dist-test-slave-4d5t"
server_key: "419e238deb335ea52a3b03a1d72bddbb"
server_key_iv: "f487a4449058ec53d4e0c98e27c4b5d0"
server_key_version: "encryptionkey@0"
I20260926 20:42:54.444720 5045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 20:42:54.449136 5060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:54.450906 5045 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:42:54.451071 5045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "d048786285b14ee19d80cc86cfe8361e"
format_stamp: "Formatted at 2026-09-26 20:42:54 on dist-test-slave-4d5t"
server_key: "419e238deb335ea52a3b03a1d72bddbb"
server_key_iv: "f487a4449058ec53d4e0c98e27c4b5d0"
server_key_version: "encryptionkey@0"
I20260926 20:42:54.451265 5045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:54.465866 5045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:54.470996 5045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:54.471328 5045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:54.493718 5045 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:41387
I20260926 20:42:54.493767 5111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:41387 every 8 connection(s)
I20260926 20:42:54.495735 5045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:42:54.500684 5112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:42:54.502300 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 5045
I20260926 20:42:54.502558 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:42:54.503414 23731 external_mini_cluster.cc:1428] Setting key 6bb409a7c119748f0011298bfd01f791
I20260926 20:42:54.516786 5112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e: Bootstrap starting.
I20260926 20:42:54.519939 5112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e: Neither blocks nor log segments found. Creating new log.
I20260926 20:42:54.521073 5112 log.cc:824] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e: Log is configured to *not* fsync() on all Append() calls
I20260926 20:42:54.524467 5112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e: No bootstrap required, opened a new log
Sep 26 20:42:54 dist-test-slave-4d5t krb5kdc[5032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:54.533572 5112 raft_consensus.cc:348] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d048786285b14ee19d80cc86cfe8361e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41387 } }
I20260926 20:42:54.534009 5112 raft_consensus.cc:374] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:42:54.534137 5112 raft_consensus.cc:729] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d048786285b14ee19d80cc86cfe8361e, State: Initialized, Role: FOLLOWER
I20260926 20:42:54.534865 5112 consensus_queue.cc:260] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d048786285b14ee19d80cc86cfe8361e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41387 } }
I20260926 20:42:54.535187 5112 raft_consensus.cc:388] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:42:54.535336 5112 raft_consensus.cc:482] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:42:54.535549 5112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:42:54.536484 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:54.505498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43642 (local address 127.23.44.254:41387)
0926 20:42:54.506390 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:54.506411 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:54.506465 (+ 54us) server_negotiation.cc:407] Connection header received
0926 20:42:54.508492 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:54.508538 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:54.509319 (+ 781us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:54.509853 (+ 534us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:54.511893 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:54.513409 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:54.515358 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:54.516069 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:54.516218 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:54.527065 (+ 10847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:54.527078 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:54.527095 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:54.527142 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:54.531932 (+ 4790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:54.532845 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:54.532855 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:54.532865 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:54.533073 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:54.533506 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:54.533511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:54.533515 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:54.534188 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:54.534511 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:54.534934 (+ 423us) server_negotiation.cc:299] Negotiation successful
0926 20:42:54.535384 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":423,"threads_started":1}
I20260926 20:42:54.538168 5112 raft_consensus.cc:504] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d048786285b14ee19d80cc86cfe8361e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41387 } }
I20260926 20:42:54.538686 5112 leader_election.cc:302] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d048786285b14ee19d80cc86cfe8361e; no voters:
I20260926 20:42:54.539376 5112 leader_election.cc:288] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:42:54.539489 5117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:42:54.540586 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:42:54.541625 5112 sys_catalog.cc:564] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:42:54.542630 5117 raft_consensus.cc:686] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [term 1 LEADER]: Becoming Leader. State: Replica: d048786285b14ee19d80cc86cfe8361e, State: Running, Role: LEADER
I20260926 20:42:54.543267 5117 consensus_queue.cc:237] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d048786285b14ee19d80cc86cfe8361e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41387 } }
I20260926 20:42:54.546893 5118 sys_catalog.cc:455] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d048786285b14ee19d80cc86cfe8361e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d048786285b14ee19d80cc86cfe8361e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41387 } } }
I20260926 20:42:54.547695 5118 sys_catalog.cc:458] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [sys.catalog]: This master's current role is: LEADER
I20260926 20:42:54.546854 5119 sys_catalog.cc:455] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d048786285b14ee19d80cc86cfe8361e. Latest consensus state: current_term: 1 leader_uuid: "d048786285b14ee19d80cc86cfe8361e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d048786285b14ee19d80cc86cfe8361e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 41387 } } }
I20260926 20:42:54.549453 5119 sys_catalog.cc:458] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e [sys.catalog]: This master's current role is: LEADER
I20260926 20:42:54.550894 5126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:42:54.556721 5126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:42:54.565830 5126 catalog_manager.cc:1347] Generated new cluster ID: d5e665a61f61495aaa4d642d2d03662c
I20260926 20:42:54.566030 5126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:42:54.579335 5126 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:42:54.580825 5126 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:42:54.589815 5126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d048786285b14ee19d80cc86cfe8361e: Generated new TSK 0
I20260926 20:42:54.591334 5126 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:42:54.602066 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 5045
I20260926 20:42:54.603367 5107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:54.697926 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 5045
2026-09-26T20:42:54Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2096 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:42:54 dist-test-slave-4d5t krb5kdc[5140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:42:54 dist-test-slave-4d5t krb5kdc[5140](info): set up 2 sockets
Sep 26 20:42:54 dist-test-slave-4d5t krb5kdc[5140](info): commencing operation
krb5kdc: starting...
W20260926 20:42:55.842250 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.008s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:42:56 dist-test-slave-4d5t krb5kdc[5140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:42:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:42:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:42:56.121313 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:45935
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:45935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:42:56.477823 5153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:56.478228 5153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:56.478407 5153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:56.485904 5153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:42:56.486089 5153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:56.486203 5153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:56.486307 5153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:42:56.486397 5153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:42:56.496579 5153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:45935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:45935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:56.498072 5153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:56.499456 5153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:56.512816 5158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:56.512887 5161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:56.512990 5159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:56.513962 5153 server_base.cc:1034] running on GCE node
I20260926 20:42:56.515347 5153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:56.517134 5153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:56.518430 5153 hybrid_clock.cc:648] HybridClock initialized: now 1790455376518369 us; error 60 us; skew 500 ppm
I20260926 20:42:56.518847 5153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:56 dist-test-slave-4d5t krb5kdc[5140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455376, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:56.522373 5153 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:42:56.524937 5153 webserver.cc:460] Webserver started at http://127.23.44.254:38347/ using document root <none> and password file <none>
I20260926 20:42:56.526051 5153 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:56.526209 5153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:42:56.526506 5153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:42:56.529947 5153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "8e4b53c0952940df9755bc5595fab053"
format_stamp: "Formatted at 2026-09-26 20:42:56 on dist-test-slave-4d5t"
server_key: "739fc0ed7b955be7403681409957d2e5"
server_key_iv: "d9b05c638c61c746ba2a644969ef994c"
server_key_version: "encryptionkey@0"
I20260926 20:42:56.530810 5153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "8e4b53c0952940df9755bc5595fab053"
format_stamp: "Formatted at 2026-09-26 20:42:56 on dist-test-slave-4d5t"
server_key: "739fc0ed7b955be7403681409957d2e5"
server_key_iv: "d9b05c638c61c746ba2a644969ef994c"
server_key_version: "encryptionkey@0"
I20260926 20:42:56.536808 5153 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260926 20:42:56.540529 5168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:56.542263 5153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:42:56.542481 5153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "8e4b53c0952940df9755bc5595fab053"
format_stamp: "Formatted at 2026-09-26 20:42:56 on dist-test-slave-4d5t"
server_key: "739fc0ed7b955be7403681409957d2e5"
server_key_iv: "d9b05c638c61c746ba2a644969ef994c"
server_key_version: "encryptionkey@0"
I20260926 20:42:56.542701 5153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:56.587039 5153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:56.593166 5153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:56.593485 5153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:56.617113 5153 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:45935
I20260926 20:42:56.617122 5219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:45935 every 8 connection(s)
I20260926 20:42:56.618716 5153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:42:56.622558 5220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:42:56.629083 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 5153
I20260926 20:42:56.629343 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:42:56.629964 23731 external_mini_cluster.cc:1428] Setting key 59b5eac751bf71cd6a1cab6ab37df8cf
I20260926 20:42:56.633905 5220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Bootstrap starting.
I20260926 20:42:56.637234 5220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Neither blocks nor log segments found. Creating new log.
I20260926 20:42:56.639071 5220 log.cc:824] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Log is configured to *not* fsync() on all Append() calls
I20260926 20:42:56.643231 5220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: No bootstrap required, opened a new log
Sep 26 20:42:56 dist-test-slave-4d5t krb5kdc[5140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:56.652350 5220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } }
I20260926 20:42:56.652853 5220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:42:56.652971 5220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e4b53c0952940df9755bc5595fab053, State: Initialized, Role: FOLLOWER
I20260926 20:42:56.653599 5220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } }
I20260926 20:42:56.653856 5220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:42:56.654013 5220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:42:56.654214 5220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:42:56.657121 5220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } }
I20260926 20:42:56.657234 5223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:56.632082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50450 (local address 127.23.44.254:45935)
0926 20:42:56.632973 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:56.632986 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:56.633031 (+ 45us) server_negotiation.cc:407] Connection header received
0926 20:42:56.634891 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:56.634915 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:56.635315 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:56.635739 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:56.637323 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:56.638838 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:56.640664 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:56.641713 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:56.641924 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:56.646737 (+ 4813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:56.646746 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:56.646796 (+ 50us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:56.646866 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:56.651470 (+ 4604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:56.652359 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:56.652367 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:56.652374 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:56.652618 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:56.653115 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:56.653122 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:56.653126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:56.653860 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:56.654285 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:56.654949 (+ 664us) server_negotiation.cc:299] Negotiation successful
0926 20:42:56.655732 (+ 783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":474,"threads_started":1}
I20260926 20:42:56.657740 5220 leader_election.cc:302] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e4b53c0952940df9755bc5595fab053; no voters:
I20260926 20:42:56.658519 5220 leader_election.cc:288] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:42:56.658759 5225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:42:56.660127 5225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 1 LEADER]: Becoming Leader. State: Replica: 8e4b53c0952940df9755bc5595fab053, State: Running, Role: LEADER
I20260926 20:42:56.661131 5220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:42:56.660991 5225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } }
I20260926 20:42:56.664332 5227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e4b53c0952940df9755bc5595fab053. Latest consensus state: current_term: 1 leader_uuid: "8e4b53c0952940df9755bc5595fab053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } } }
I20260926 20:42:56.664492 5226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e4b53c0952940df9755bc5595fab053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } } }
I20260926 20:42:56.664824 5227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: This master's current role is: LEADER
I20260926 20:42:56.664958 5226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: This master's current role is: LEADER
I20260926 20:42:56.666018 5232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:42:56.673303 5232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:42:56.685350 5232 catalog_manager.cc:1347] Generated new cluster ID: e76c028afacd4f968c57ec2efab64020
I20260926 20:42:56.685526 5232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:42:56.697990 5232 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:42:56.700296 5232 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:42:56.708642 5232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Generated new TSK 0
I20260926 20:42:56.709770 5232 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:42:56.770570 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45935
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:42:57.138609 5246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:57.138918 5246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:57.139014 5246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:57.146490 5246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:57.146618 5246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:57.146734 5246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:42:57.156939 5246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:57.158375 5246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:57.159762 5246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:57.174943 5252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:57.175050 5251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:57.175046 5254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:57.176251 5246 server_base.cc:1034] running on GCE node
I20260926 20:42:57.177018 5246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:57.178088 5246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:57.179344 5246 hybrid_clock.cc:648] HybridClock initialized: now 1790455377179273 us; error 62 us; skew 500 ppm
I20260926 20:42:57.179713 5246 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:57 dist-test-slave-4d5t krb5kdc[5140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455377, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:57.183581 5246 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:42:57.186070 5246 webserver.cc:460] Webserver started at http://127.23.44.193:33601/ using document root <none> and password file <none>
I20260926 20:42:57.186859 5246 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:57.187027 5246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:42:57.187340 5246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:42:57.190624 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "aad6447897c84bbe8edb1874bb20c162"
format_stamp: "Formatted at 2026-09-26 20:42:57 on dist-test-slave-4d5t"
server_key: "9779adf0f92a557f41af8e643f1b94d1"
server_key_iv: "740dd5990cef3485aabc135393f67c01"
server_key_version: "encryptionkey@0"
I20260926 20:42:57.191584 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "aad6447897c84bbe8edb1874bb20c162"
format_stamp: "Formatted at 2026-09-26 20:42:57 on dist-test-slave-4d5t"
server_key: "9779adf0f92a557f41af8e643f1b94d1"
server_key_iv: "740dd5990cef3485aabc135393f67c01"
server_key_version: "encryptionkey@0"
I20260926 20:42:57.197559 5246 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260926 20:42:57.201431 5261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:57.202888 5246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:42:57.203104 5246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "aad6447897c84bbe8edb1874bb20c162"
format_stamp: "Formatted at 2026-09-26 20:42:57 on dist-test-slave-4d5t"
server_key: "9779adf0f92a557f41af8e643f1b94d1"
server_key_iv: "740dd5990cef3485aabc135393f67c01"
server_key_version: "encryptionkey@0"
I20260926 20:42:57.203333 5246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:57.219338 5246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:57.224460 5246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:57.225126 5246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:57.226430 5246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:42:57.228533 5246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:42:57.228721 5246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260926 20:42:57.228873 5246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:42:57.228924 5246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:57.267025 5246 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:38609
I20260926 20:42:57.267072 5373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:38609 every 8 connection(s)
I20260926 20:42:57.268388 5246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:42:57.270183 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 5246
I20260926 20:42:57.270419 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:42:57.271476 23731 external_mini_cluster.cc:1428] Setting key bd5387dad3007f556b85a44e1531befb
Sep 26 20:42:57 dist-test-slave-4d5t krb5kdc[5140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455377, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:57.292161 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:57.271784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:50753 (local address 127.23.44.254:45935)
0926 20:42:57.272247 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:57.272252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:57.273683 (+ 1431us) server_negotiation.cc:407] Connection header received
0926 20:42:57.275467 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:57.275472 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:57.275564 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:57.275681 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:57.277335 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:57.278558 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:57.280323 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:57.280868 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:57.280981 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:57.285698 (+ 4717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:57.285703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:57.285706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:57.285741 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:57.288489 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:57.289656 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:57.289661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:57.289664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:57.289795 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:57.290311 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:57.290314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:57.290316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:57.290594 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:57.290707 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:57.291693 (+ 986us) server_negotiation.cc:299] Negotiation successful
0926 20:42:57.291887 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":252,"threads_started":1}
I20260926 20:42:57.293892 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:57.272464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:50753)
0926 20:42:57.273407 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:57.273461 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:57.275189 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:57.275925 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:57.275934 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:57.276457 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:57.277147 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:57.277165 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:57.278795 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:57.278802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:57.280131 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:57.280152 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:57.281155 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:57.281165 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:57.281508 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:57.282361 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:57.282388 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:57.285391 (+ 3003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:57.288698 (+ 3307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:57.288712 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:57.288735 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:57.289463 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:57.289915 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:57.289923 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:57.289928 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:57.290186 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:57.290732 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:57.290744 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:57.291238 (+ 494us) client_negotiation.cc:769] Sending connection context
0926 20:42:57.291575 (+ 337us) client_negotiation.cc:240] Negotiation successful
0926 20:42:57.292218 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":325,"threads_started":1}
I20260926 20:42:57.295866 5374 heartbeater.cc:347] Connected to a master server at 127.23.44.254:45935
I20260926 20:42:57.296221 5374 heartbeater.cc:464] Registering TS with master...
I20260926 20:42:57.297077 5374 heartbeater.cc:511] Master 127.23.44.254:45935 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
I20260926 20:42:57.299364 5185 ts_manager.cc:194] Registered new tserver with Master: aad6447897c84bbe8edb1874bb20c162 (127.23.44.193:38609)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
I20260926 20:42:57.301096 5185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:50753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:42:57.329239 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45935
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:42:57.679570 5380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:57.679886 5380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:57.679991 5380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:57.687484 5380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:57.687623 5380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:57.687742 5380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:42:57.698194 5380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:57.699522 5380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:57.700917 5380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:57.715273 5385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:57.715544 5388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:57.715243 5386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:57.716210 5380 server_base.cc:1034] running on GCE node
I20260926 20:42:57.716890 5380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:57.717862 5380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:57.719205 5380 hybrid_clock.cc:648] HybridClock initialized: now 1790455377719139 us; error 59 us; skew 500 ppm
I20260926 20:42:57.719563 5380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:57 dist-test-slave-4d5t krb5kdc[5140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455377, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:57.723425 5380 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:42:57.725914 5380 webserver.cc:460] Webserver started at http://127.23.44.194:36191/ using document root <none> and password file <none>
I20260926 20:42:57.726929 5380 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:57.727134 5380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:42:57.727466 5380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:42:57.730573 5380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "5bd22737c91d42c498f6643804dd1d71"
format_stamp: "Formatted at 2026-09-26 20:42:57 on dist-test-slave-4d5t"
server_key: "0e36ac92fdf836fe701abefd08e5de6d"
server_key_iv: "8fee978216874a591913cff4bd7f84d3"
server_key_version: "encryptionkey@0"
I20260926 20:42:57.731369 5380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "5bd22737c91d42c498f6643804dd1d71"
format_stamp: "Formatted at 2026-09-26 20:42:57 on dist-test-slave-4d5t"
server_key: "0e36ac92fdf836fe701abefd08e5de6d"
server_key_iv: "8fee978216874a591913cff4bd7f84d3"
server_key_version: "encryptionkey@0"
I20260926 20:42:57.737151 5380 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 20:42:57.741461 5395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:57.743180 5380 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:42:57.743412 5380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "5bd22737c91d42c498f6643804dd1d71"
format_stamp: "Formatted at 2026-09-26 20:42:57 on dist-test-slave-4d5t"
server_key: "0e36ac92fdf836fe701abefd08e5de6d"
server_key_iv: "8fee978216874a591913cff4bd7f84d3"
server_key_version: "encryptionkey@0"
I20260926 20:42:57.743645 5380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:57.765733 5380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:57.770642 5380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:57.770958 5380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:57.772038 5380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:42:57.773599 5380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:42:57.773706 5380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:57.773872 5380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:42:57.773944 5380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:57.810966 5380 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:36159
I20260926 20:42:57.810984 5507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:36159 every 8 connection(s)
I20260926 20:42:57.812352 5380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:42:57.815251 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 5380
I20260926 20:42:57.815482 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:42:57.815948 23731 external_mini_cluster.cc:1428] Setting key 241c86b8d7d21cd45a3094d722cff447
Sep 26 20:42:57 dist-test-slave-4d5t krb5kdc[5140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455377, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:57.836381 5510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:57.815087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:42221 (local address 127.23.44.254:45935)
0926 20:42:57.815611 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:57.815618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:57.816623 (+ 1005us) server_negotiation.cc:407] Connection header received
0926 20:42:57.818778 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:57.818784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:57.818864 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:57.819006 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:57.820841 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:57.822032 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:57.824453 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:57.825254 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:57.825416 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:57.830150 (+ 4734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:57.830155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:57.830158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:57.830195 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:57.833054 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:57.833893 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:57.833898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:57.833903 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:57.834039 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:57.834490 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:57.834497 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:57.834506 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:57.834828 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:57.834994 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:57.835743 (+ 749us) server_negotiation.cc:299] Negotiation successful
0926 20:42:57.835983 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":277,"threads_started":1}
I20260926 20:42:57.837472 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:57.815581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:42221)
0926 20:42:57.816429 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:57.816481 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:57.818452 (+ 1971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:57.819215 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:57.819223 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:57.819984 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:57.820647 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:57.820668 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:57.822349 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:57.822356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:57.824212 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:57.824235 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:57.825598 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:57.825606 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:57.825866 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:57.826816 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:57.826860 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:57.829844 (+ 2984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:57.833244 (+ 3400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:57.833253 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:57.833269 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:57.833746 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:57.834159 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:57.834165 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:57.834168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:57.834363 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:57.834972 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:57.834980 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:57.835368 (+ 388us) client_negotiation.cc:769] Sending connection context
0926 20:42:57.835654 (+ 286us) client_negotiation.cc:240] Negotiation successful
0926 20:42:57.836255 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":352,"threads_started":1}
I20260926 20:42:57.839184 5508 heartbeater.cc:347] Connected to a master server at 127.23.44.254:45935
I20260926 20:42:57.839553 5508 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
I20260926 20:42:57.840610 5508 heartbeater.cc:511] Master 127.23.44.254:45935 requested a full tablet report, sending...
I20260926 20:42:57.842345 5185 ts_manager.cc:194] Registered new tserver with Master: 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159)
I20260926 20:42:57.843645 5185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:42221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:42:57.880833 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45935
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:42:58.230598 5514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:58.230979 5514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:58.231163 5514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:58.238941 5514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:58.239125 5514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:58.239316 5514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:42:58.249995 5514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:58.251349 5514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:58.252780 5514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:58.267318 5519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:58.267285 5520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:58.267740 5522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:58.268345 5514 server_base.cc:1034] running on GCE node
I20260926 20:42:58.269626 5514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:58.270725 5514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:58.272035 5514 hybrid_clock.cc:648] HybridClock initialized: now 1790455378271979 us; error 47 us; skew 500 ppm
I20260926 20:42:58.272392 5514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:58 dist-test-slave-4d5t krb5kdc[5140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455378, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:58.275751 5514 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:42:58.277366 5514 webserver.cc:460] Webserver started at http://127.23.44.195:33429/ using document root <none> and password file <none>
I20260926 20:42:58.278090 5514 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:58.278246 5514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:42:58.278554 5514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:42:58.281647 5514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "387b20d842b6457b9ab32a65fd6c447c"
format_stamp: "Formatted at 2026-09-26 20:42:58 on dist-test-slave-4d5t"
server_key: "85d5a20e5a8218e834f4941a86d125fc"
server_key_iv: "3f31d2db53973ec96debc462d5d10936"
server_key_version: "encryptionkey@0"
I20260926 20:42:58.282486 5514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "387b20d842b6457b9ab32a65fd6c447c"
format_stamp: "Formatted at 2026-09-26 20:42:58 on dist-test-slave-4d5t"
server_key: "85d5a20e5a8218e834f4941a86d125fc"
server_key_iv: "3f31d2db53973ec96debc462d5d10936"
server_key_version: "encryptionkey@0"
I20260926 20:42:58.288138 5514 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260926 20:42:58.292912 5529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:58.294296 5514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260926 20:42:58.294493 5514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "387b20d842b6457b9ab32a65fd6c447c"
format_stamp: "Formatted at 2026-09-26 20:42:58 on dist-test-slave-4d5t"
server_key: "85d5a20e5a8218e834f4941a86d125fc"
server_key_iv: "3f31d2db53973ec96debc462d5d10936"
server_key_version: "encryptionkey@0"
I20260926 20:42:58.294730 5514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:58.304252 5374 heartbeater.cc:503] Master 127.23.44.254:45935 was elected leader, sending a full tablet report...
I20260926 20:42:58.318181 5514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:58.322976 5514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:58.323289 5514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:58.324309 5514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:42:58.325774 5514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:42:58.325886 5514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:58.326042 5514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:42:58.326117 5514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:58.362924 5514 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:38619
I20260926 20:42:58.362964 5641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:38619 every 8 connection(s)
I20260926 20:42:58.364255 5514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:42:58.367594 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 5514
I20260926 20:42:58.367836 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:42:58.368501 23731 external_mini_cluster.cc:1428] Setting key afff882470a832c21edebe30acfb0fd6
Sep 26 20:42:58 dist-test-slave-4d5t krb5kdc[5140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455378, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:42:58.385614 5644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.366919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:55947 (local address 127.23.44.254:45935)
0926 20:42:58.367409 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:58.367414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:58.368484 (+ 1070us) server_negotiation.cc:407] Connection header received
0926 20:42:58.371033 (+ 2549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:58.371038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:58.371126 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:58.371242 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:58.372951 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.373754 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.375308 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.375697 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.375797 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.379351 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:58.379356 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:58.379359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:58.379391 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:58.382186 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:58.383148 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:58.383152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:58.383155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:58.383268 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:58.383866 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:58.383870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:58.383872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:58.384234 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:58.384354 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:58.385164 (+ 810us) server_negotiation.cc:299] Negotiation successful
0926 20:42:58.385336 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":264,"threads_started":1}
I20260926 20:42:58.386521 5645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.367382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:55947)
0926 20:42:58.368281 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:58.368359 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:58.370712 (+ 2353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:58.371510 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:58.371523 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:58.372118 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:42:58.372751 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:58.372770 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.373906 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.373910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:58.375165 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:58.375177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.375926 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.375931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:58.376126 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.376875 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:42:58.376900 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:42:58.379132 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:42:58.382379 (+ 3247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:58.382387 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:58.382403 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:58.383039 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:58.383464 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:42:58.383468 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:42:58.383470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:42:58.383746 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:42:58.384349 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:42:58.384355 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:42:58.384750 (+ 395us) client_negotiation.cc:769] Sending connection context
0926 20:42:58.384986 (+ 236us) client_negotiation.cc:240] Negotiation successful
0926 20:42:58.385387 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":353,"threads_started":1}
I20260926 20:42:58.388222 5642 heartbeater.cc:347] Connected to a master server at 127.23.44.254:45935
I20260926 20:42:58.388608 5642 heartbeater.cc:464] Registering TS with master...
I20260926 20:42:58.389359 5642 heartbeater.cc:511] Master 127.23.44.254:45935 requested a full tablet report, sending...
I20260926 20:42:58.390950 5185 ts_manager.cc:194] Registered new tserver with Master: 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619)
I20260926 20:42:58.391814 5185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:55947
I20260926 20:42:58.397424 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:42:58.416885 5644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.402803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.23.44.254:45935)
0926 20:42:58.403055 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:58.403064 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:58.403388 (+ 324us) server_negotiation.cc:407] Connection header received
0926 20:42:58.403524 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:58.403528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:58.403595 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:58.403679 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:58.405183 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.405936 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.407344 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.407793 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.407911 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.411052 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:58.411057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:58.411060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:58.411099 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:58.413847 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:58.414563 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:58.414567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:58.414570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:58.414692 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:58.415450 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:58.415454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:58.415457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:58.415826 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:58.416015 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:58.416343 (+ 328us) server_negotiation.cc:299] Negotiation successful
0926 20:42:58.416527 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 20:42:58.421360 5185 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:42:58.428354 5185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:42:58.468885 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.442992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.23.44.195:38619)
0926 20:42:58.460446 (+ 17454us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:58.460454 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:58.460485 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:42:58.460615 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:58.460622 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:58.461518 (+ 896us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:58.461716 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:42:58.462314 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.463705 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.465749 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.467210 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.467346 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.468066 (+ 720us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:58.468279 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 20:42:58.468460 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17284,"thread_start_us":17096,"threads_started":1}
I20260926 20:42:58.469069 5658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.442620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:38619 (local address 127.0.0.1:44244)
0926 20:42:58.458240 (+ 15620us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:58.458261 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:58.458385 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:58.461739 (+ 3354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:58.461744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:58.461769 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:42:58.462115 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:58.462128 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.463854 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.463860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:58.465601 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:58.465614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.467585 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.467593 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:58.467812 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.467869 (+ 57us) client_negotiation.cc:769] Sending connection context
0926 20:42:58.468247 (+ 378us) client_negotiation.cc:240] Negotiation successful
0926 20:42:58.468465 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15468,"mutex_wait_us":55,"thread_start_us":15313,"threads_started":1}
I20260926 20:42:58.469094 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.441983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:38609 (local address 127.0.0.1:35198)
0926 20:42:58.449454 (+ 7471us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:58.449529 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:58.449778 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:58.460632 (+ 10854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:58.460639 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:58.460663 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:42:58.461141 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:58.461154 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.463142 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.463147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:58.465942 (+ 2795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:58.465951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.467789 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.467796 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:58.468108 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.468275 (+ 167us) client_negotiation.cc:769] Sending connection context
0926 20:42:58.468351 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 20:42:58.468537 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7094,"mutex_wait_us":237,"thread_start_us":6900,"threads_started":1}
I20260926 20:42:58.469642 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.443479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60814 (local address 127.23.44.194:36159)
0926 20:42:58.459169 (+ 15690us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:58.459183 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:58.459225 (+ 42us) server_negotiation.cc:407] Connection header received
0926 20:42:58.459367 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:58.459380 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:58.459755 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:58.460101 (+ 346us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:42:58.461134 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.463087 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.466093 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.467570 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.467706 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.467904 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:58.469118 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0926 20:42:58.469300 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15532,"thread_start_us":15415,"threads_started":1}
I20260926 20:42:58.469898 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.442427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:36159 (local address 127.0.0.1:60814)
0926 20:42:58.457466 (+ 15039us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:58.457497 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:58.457677 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:58.460389 (+ 2712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:58.460396 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:58.460438 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:42:58.460920 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:58.460935 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.463371 (+ 2436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.463382 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:58.465906 (+ 2524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:58.465924 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.467802 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.467807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:58.468106 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.468276 (+ 170us) client_negotiation.cc:769] Sending connection context
0926 20:42:58.468353 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 20:42:58.468528 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14818,"mutex_wait_us":196,"thread_start_us":16065,"threads_started":1}
I20260926 20:42:58.470068 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.442167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35198 (local address 127.23.44.193:38609)
0926 20:42:58.460031 (+ 17864us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:58.460041 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:58.460072 (+ 31us) server_negotiation.cc:407] Connection header received
0926 20:42:58.460171 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:58.460180 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:58.460399 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:58.460618 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:42:58.461402 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.462993 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.466134 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.467517 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.469183 (+ 1666us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.469414 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:58.469499 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 20:42:58.469671 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17668,"thread_start_us":17528,"threads_started":1}
I20260926 20:42:58.473595 5443 tablet_service.cc:1467] Processing CreateTablet for tablet 1eef9f290b7a48a29c6cc124c525bf2d (DEFAULT_TABLE table=test-table [id=ca6eef09230a476e8349fa116dbb34a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:42:58.473619 5309 tablet_service.cc:1467] Processing CreateTablet for tablet 1eef9f290b7a48a29c6cc124c525bf2d (DEFAULT_TABLE table=test-table [id=ca6eef09230a476e8349fa116dbb34a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:42:58.473619 5577 tablet_service.cc:1467] Processing CreateTablet for tablet 1eef9f290b7a48a29c6cc124c525bf2d (DEFAULT_TABLE table=test-table [id=ca6eef09230a476e8349fa116dbb34a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:42:58.474901 5443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eef9f290b7a48a29c6cc124c525bf2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:42:58.475006 5577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eef9f290b7a48a29c6cc124c525bf2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:42:58.475013 5309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1eef9f290b7a48a29c6cc124c525bf2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:42:58.487215 5661 tablet_bootstrap.cc:492] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71: Bootstrap starting.
I20260926 20:42:58.489869 5662 tablet_bootstrap.cc:492] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162: Bootstrap starting.
I20260926 20:42:58.490713 5663 tablet_bootstrap.cc:492] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c: Bootstrap starting.
I20260926 20:42:58.491233 5661 tablet_bootstrap.cc:654] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71: Neither blocks nor log segments found. Creating new log.
I20260926 20:42:58.492687 5661 log.cc:824] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71: Log is configured to *not* fsync() on all Append() calls
I20260926 20:42:58.494508 5663 tablet_bootstrap.cc:654] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c: Neither blocks nor log segments found. Creating new log.
I20260926 20:42:58.495787 5663 log.cc:824] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c: Log is configured to *not* fsync() on all Append() calls
I20260926 20:42:58.496006 5662 tablet_bootstrap.cc:654] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162: Neither blocks nor log segments found. Creating new log.
I20260926 20:42:58.496165 5661 tablet_bootstrap.cc:492] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71: No bootstrap required, opened a new log
I20260926 20:42:58.496462 5661 ts_tablet_manager.cc:1397] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 20:42:58.497241 5662 log.cc:824] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162: Log is configured to *not* fsync() on all Append() calls
I20260926 20:42:58.498961 5663 tablet_bootstrap.cc:492] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c: No bootstrap required, opened a new log
I20260926 20:42:58.499235 5663 ts_tablet_manager.cc:1397] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 20:42:58.500452 5662 tablet_bootstrap.cc:492] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162: No bootstrap required, opened a new log
I20260926 20:42:58.500805 5662 ts_tablet_manager.cc:1397] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260926 20:42:58.507061 5663 raft_consensus.cc:348] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:42:58.507481 5663 raft_consensus.cc:374] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:42:58.507615 5663 raft_consensus.cc:729] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 387b20d842b6457b9ab32a65fd6c447c, State: Initialized, Role: FOLLOWER
I20260926 20:42:58.507454 5661 raft_consensus.cc:348] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:42:58.507880 5661 raft_consensus.cc:374] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:42:58.508104 5661 raft_consensus.cc:729] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bd22737c91d42c498f6643804dd1d71, State: Initialized, Role: FOLLOWER
I20260926 20:42:58.508324 5663 consensus_queue.cc:260] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:42:58.508738 5661 consensus_queue.cc:260] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:42:58.509721 5642 heartbeater.cc:503] Master 127.23.44.254:45935 was elected leader, sending a full tablet report...
I20260926 20:42:58.510492 5663 ts_tablet_manager.cc:1428] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260926 20:42:58.511389 5508 heartbeater.cc:503] Master 127.23.44.254:45935 was elected leader, sending a full tablet report...
I20260926 20:42:58.512310 5662 raft_consensus.cc:348] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:42:58.512773 5662 raft_consensus.cc:374] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:42:58.512869 5662 raft_consensus.cc:729] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad6447897c84bbe8edb1874bb20c162, State: Initialized, Role: FOLLOWER
I20260926 20:42:58.513458 5662 consensus_queue.cc:260] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:42:58.513834 5661 ts_tablet_manager.cc:1428] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
I20260926 20:42:58.515654 5662 ts_tablet_manager.cc:1428] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260926 20:42:58.521263 5375 tablet.cc:2369] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:42:58.564548 5509 tablet.cc:2369] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:42:58.616003 5643 tablet.cc:2369] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:42:58.630645 5669 raft_consensus.cc:482] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:42:58.631001 5669 raft_consensus.cc:504] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:42:58.632800 5669 leader_election.cc:288] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159), 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619)
I20260926 20:42:58.639647 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.633407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:43563 (local address 127.23.44.195:38619)
0926 20:42:58.633652 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:58.633661 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:58.634298 (+ 637us) server_negotiation.cc:407] Connection header received
0926 20:42:58.634517 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:58.634521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:58.634580 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:58.634685 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:42:58.635339 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.636061 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.637946 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.638701 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.638898 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.638973 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:58.639244 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 20:42:58.639376 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 20:42:58.639762 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.633151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:37373 (local address 127.23.44.194:36159)
0926 20:42:58.633390 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:58.633399 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:58.634091 (+ 692us) server_negotiation.cc:407] Connection header received
0926 20:42:58.634338 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:58.634343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:58.634412 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:58.634513 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:42:58.635190 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.635917 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.637967 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.638779 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.638959 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.639043 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:58.639331 (+ 288us) server_negotiation.cc:299] Negotiation successful
0926 20:42:58.639495 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:42:58.639886 5670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.633107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:36159 (local address 127.23.44.193:37373)
0926 20:42:58.633966 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:58.633983 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:58.634181 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:58.634534 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:58.634538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:58.634563 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:42:58.635040 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:58.635051 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.636057 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.636063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:58.637792 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:58.637805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.639022 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.639029 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:58.639171 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.639193 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 20:42:58.639293 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 20:42:58.639486 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"mutex_wait_us":161,"thread_start_us":486,"threads_started":1}
I20260926 20:42:58.640250 5671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.633215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:38619 (local address 127.23.44.193:43563)
0926 20:42:58.634182 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0926 20:42:58.634200 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:42:58.634372 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:42:58.634704 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:42:58.634708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:42:58.634734 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:42:58.635168 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:58.635177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.636207 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.636216 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:58.637793 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:42:58.637805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.638916 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.638923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:42:58.639085 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.639111 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 20:42:58.639218 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 20:42:58.639385 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"mutex_wait_us":23,"thread_start_us":657,"threads_started":1}
I20260926 20:42:58.640978 5463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eef9f290b7a48a29c6cc124c525bf2d" candidate_uuid: "aad6447897c84bbe8edb1874bb20c162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bd22737c91d42c498f6643804dd1d71" is_pre_election: true
I20260926 20:42:58.641198 5597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eef9f290b7a48a29c6cc124c525bf2d" candidate_uuid: "aad6447897c84bbe8edb1874bb20c162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "387b20d842b6457b9ab32a65fd6c447c" is_pre_election: true
I20260926 20:42:58.641537 5463 raft_consensus.cc:2454] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aad6447897c84bbe8edb1874bb20c162 in term 0.
I20260926 20:42:58.641695 5597 raft_consensus.cc:2454] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aad6447897c84bbe8edb1874bb20c162 in term 0.
I20260926 20:42:58.642246 5263 leader_election.cc:302] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bd22737c91d42c498f6643804dd1d71, aad6447897c84bbe8edb1874bb20c162; no voters:
I20260926 20:42:58.642725 5669 raft_consensus.cc:2781] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:42:58.642899 5669 raft_consensus.cc:482] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:42:58.643029 5669 raft_consensus.cc:3037] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:42:58.645269 5669 raft_consensus.cc:504] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:42:58.645947 5669 leader_election.cc:288] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 1 election: Requested vote from peers 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159), 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619)
I20260926 20:42:58.646486 5597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eef9f290b7a48a29c6cc124c525bf2d" candidate_uuid: "aad6447897c84bbe8edb1874bb20c162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "387b20d842b6457b9ab32a65fd6c447c"
I20260926 20:42:58.646718 5597 raft_consensus.cc:3037] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:42:58.646821 5463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1eef9f290b7a48a29c6cc124c525bf2d" candidate_uuid: "aad6447897c84bbe8edb1874bb20c162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bd22737c91d42c498f6643804dd1d71"
I20260926 20:42:58.647054 5463 raft_consensus.cc:3037] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:42:58.650141 5597 raft_consensus.cc:2454] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aad6447897c84bbe8edb1874bb20c162 in term 1.
I20260926 20:42:58.650278 5463 raft_consensus.cc:2454] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aad6447897c84bbe8edb1874bb20c162 in term 1.
I20260926 20:42:58.650627 5262 leader_election.cc:302] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 387b20d842b6457b9ab32a65fd6c447c, aad6447897c84bbe8edb1874bb20c162; no voters:
I20260926 20:42:58.651002 5669 raft_consensus.cc:2781] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:42:58.651439 5669 raft_consensus.cc:686] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [term 1 LEADER]: Becoming Leader. State: Replica: aad6447897c84bbe8edb1874bb20c162, State: Running, Role: LEADER
I20260926 20:42:58.652019 5669 consensus_queue.cc:237] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:42:58.656893 5184 catalog_manager.cc:5495] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 reported cstate change: term changed from 0 to 1, leader changed from <none> to aad6447897c84bbe8edb1874bb20c162 (127.23.44.193). New cstate: current_term: 1 leader_uuid: "aad6447897c84bbe8edb1874bb20c162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } health_report { overall_health: UNKNOWN } } }
I20260926 20:42:58.709602 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.703517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35212 (local address 127.23.44.193:38609)
0926 20:42:58.703857 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:58.703862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:58.703883 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:42:58.703985 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:58.703989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:58.704077 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:58.704332 (+ 255us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:42:58.704709 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.705714 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.707175 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.707627 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.707735 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.708278 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:42:58.708948 (+ 670us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:42:58.709157 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:58.709311 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 20:42:58.709387 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260926 20:42:58.718191 5597 raft_consensus.cc:1261] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c [term 1 FOLLOWER]: Refusing update from remote peer aad6447897c84bbe8edb1874bb20c162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:42:58.718256 5463 raft_consensus.cc:1261] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71 [term 1 FOLLOWER]: Refusing update from remote peer aad6447897c84bbe8edb1874bb20c162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:42:58.719241 5672 consensus_queue.cc:1035] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:42:58.719631 5669 consensus_queue.cc:1035] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:42:58.747655 5184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:42:58.747989 5184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:42:58.752485 5184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Sending DeleteTablet for 3 replicas of tablet 1eef9f290b7a48a29c6cc124c525bf2d
I20260926 20:42:58.753832 5309 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eef9f290b7a48a29c6cc124c525bf2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:42:58 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:35198
I20260926 20:42:58.753859 5443 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eef9f290b7a48a29c6cc124c525bf2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:42:58 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:60814
I20260926 20:42:58.754535 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 5246
I20260926 20:42:58.754621 5577 tablet_service.cc:1514] Processing DeleteTablet for tablet 1eef9f290b7a48a29c6cc124c525bf2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:42:58 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:44244
I20260926 20:42:58.755717 5687 tablet_replica.cc:330] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c: stopping tablet replica
I20260926 20:42:58.756296 5687 raft_consensus.cc:2229] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:42:58.756945 5687 raft_consensus.cc:2258] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:42:58.757862 5685 tablet_replica.cc:330] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71: stopping tablet replica
I20260926 20:42:58.758399 5685 raft_consensus.cc:2229] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:42:58.759025 5685 raft_consensus.cc:2258] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:42:58.759959 5686 tablet_replica.cc:330] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162: stopping tablet replica
I20260926 20:42:58.760390 5686 raft_consensus.cc:2229] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:42:58.761198 5686 raft_consensus.cc:2258] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:42:58.762248 5687 ts_tablet_manager.cc:1905] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:42:58.763469 5686 ts_tablet_manager.cc:1905] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:42:58.770169 5685 ts_tablet_manager.cc:1905] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:42:58.770834 5687 ts_tablet_manager.cc:1918] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:42:58.771039 5687 log.cc:1196] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/1eef9f290b7a48a29c6cc124c525bf2d
I20260926 20:42:58.771587 5687 ts_tablet_manager.cc:1939] T 1eef9f290b7a48a29c6cc124c525bf2d P 387b20d842b6457b9ab32a65fd6c447c: Deleting consensus metadata
I20260926 20:42:58.773736 5169 catalog_manager.cc:4841] TS 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619): tablet 1eef9f290b7a48a29c6cc124c525bf2d (table test-table [id=ca6eef09230a476e8349fa116dbb34a6]) successfully deleted
I20260926 20:42:58.775220 5686 ts_tablet_manager.cc:1918] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:42:58.775453 5686 log.cc:1196] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/1eef9f290b7a48a29c6cc124c525bf2d
Sep 26 20:42:58 dist-test-slave-4d5t krb5kdc[5140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:42:58.776034 5686 ts_tablet_manager.cc:1939] T 1eef9f290b7a48a29c6cc124c525bf2d P aad6447897c84bbe8edb1874bb20c162: Deleting consensus metadata
I20260926 20:42:58.777999 5171 catalog_manager.cc:4841] TS aad6447897c84bbe8edb1874bb20c162 (127.23.44.193:38609): tablet 1eef9f290b7a48a29c6cc124c525bf2d (table test-table [id=ca6eef09230a476e8349fa116dbb34a6]) successfully deleted
I20260926 20:42:58.782811 5685 ts_tablet_manager.cc:1918] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:42:58.783018 5685 log.cc:1196] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/1eef9f290b7a48a29c6cc124c525bf2d
I20260926 20:42:58.783567 5685 ts_tablet_manager.cc:1939] T 1eef9f290b7a48a29c6cc124c525bf2d P 5bd22737c91d42c498f6643804dd1d71: Deleting consensus metadata
I20260926 20:42:58.785473 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.755761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35216 (local address 127.23.44.193:38609)
0926 20:42:58.755953 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:58.755959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:58.764198 (+ 8239us) server_negotiation.cc:407] Connection header received
0926 20:42:58.764410 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:58.764416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:58.764506 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:58.764669 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:58.766668 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.768041 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.770263 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.771010 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.771117 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.777334 (+ 6217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:58.777350 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:58.777368 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:58.777428 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:58.781708 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:58.782738 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:58.782750 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:58.782759 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:58.782925 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:58.783411 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:58.783416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:58.783419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:58.783908 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:58.784238 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:58.784610 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 20:42:58.784872 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 20:42:58.785548 5170 catalog_manager.cc:4841] TS 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159): tablet 1eef9f290b7a48a29c6cc124c525bf2d (table test-table [id=ca6eef09230a476e8349fa116dbb34a6]) successfully deleted
I20260926 20:42:58.786163 5369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:58.886209 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 5246
I20260926 20:42:58.907393 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 5380
Sep 26 20:42:58 dist-test-slave-4d5t krb5kdc[5140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:42:58.924084 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:58.908182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60820 (local address 127.23.44.194:36159)
0926 20:42:58.908353 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:58.908358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:58.908408 (+ 50us) server_negotiation.cc:407] Connection header received
0926 20:42:58.908629 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:58.908633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:58.908723 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:58.908812 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:58.910050 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.910801 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.912151 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:58.912549 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:58.912760 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:58.917422 (+ 4662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:58.917455 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:58.917486 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:58.917577 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:58.921539 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:58.922244 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:58.922250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:58.922253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:58.922385 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:58.922740 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:58.922743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:58.922745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:58.923141 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:58.923356 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:58.923642 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 20:42:58.923812 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 20:42:58.924882 5503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:59.015842 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 5380
I20260926 20:42:59.033064 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 5514
Sep 26 20:42:59 dist-test-slave-4d5t krb5kdc[5140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:42:59.050107 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:42:59.033868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44248 (local address 127.23.44.195:38619)
0926 20:42:59.034071 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 20:42:59.034077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:42:59.034099 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:42:59.034169 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:42:59.034174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:42:59.034260 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:42:59.034376 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:42:59.035774 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:59.036501 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:59.037803 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:42:59.038219 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:42:59.038368 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:42:59.043445 (+ 5077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:42:59.043463 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:42:59.043483 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:42:59.043526 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:42:59.046898 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:59.047845 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:59.047853 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:59.047856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:59.048007 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:42:59.048388 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:42:59.048394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:42:59.048397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:42:59.048861 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:42:59.049182 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0926 20:42:59.049483 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 20:42:59.049718 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:42:59.051035 5637 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:59.150805 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 5514
I20260926 20:42:59.170288 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 5153
I20260926 20:42:59.171479 5215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:42:59.287220 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 5153
I20260926 20:42:59.310267 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:45935
--webserver_interface=127.23.44.254
--webserver_port=38347
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:45935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:42:59.667698 5693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:42:59.668048 5693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:42:59.668227 5693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:42:59.676196 5693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:42:59.676381 5693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:42:59.676523 5693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:42:59.676827 5693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:42:59.676971 5693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:42:59.688848 5693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:45935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:45935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=38347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:42:59.690316 5693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:42:59.691706 5693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:42:59.716374 5700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:59.716346 5699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:42:59.716583 5702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:42:59.717510 5693 server_base.cc:1034] running on GCE node
I20260926 20:42:59.718480 5693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:42:59.720238 5693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:42:59.721524 5693 hybrid_clock.cc:648] HybridClock initialized: now 1790455379721444 us; error 67 us; skew 500 ppm
I20260926 20:42:59.722004 5693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:42:59 dist-test-slave-4d5t krb5kdc[5140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455379, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:42:59.729436 5693 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:42:59.731648 5693 webserver.cc:460] Webserver started at http://127.23.44.254:38347/ using document root <none> and password file <none>
I20260926 20:42:59.732496 5693 fs_manager.cc:359] Metadata directory not provided
I20260926 20:42:59.732712 5693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:42:59.739569 5693 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260926 20:42:59.742970 5709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:42:59.744904 5693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:42:59.745146 5693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "8e4b53c0952940df9755bc5595fab053"
format_stamp: "Formatted at 2026-09-26 20:42:56 on dist-test-slave-4d5t"
server_key: "739fc0ed7b955be7403681409957d2e5"
server_key_iv: "d9b05c638c61c746ba2a644969ef994c"
server_key_version: "encryptionkey@0"
I20260926 20:42:59.746387 5693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:42:59.764317 5693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:42:59.769133 5693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:42:59.769454 5693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:42:59.791621 5693 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:45935
I20260926 20:42:59.791657 5760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:45935 every 8 connection(s)
I20260926 20:42:59.793494 5693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:42:59.798703 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 5693
I20260926 20:42:59.799714 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:38609
--local_ip_for_outbound_sockets=127.23.44.193
--tserver_master_addrs=127.23.44.254:45935
--webserver_port=33601
--webserver_interface=127.23.44.193
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:42:59.801966 5761 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:42:59.807976 5761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Bootstrap starting.
I20260926 20:42:59.830003 5761 log.cc:824] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Log is configured to *not* fsync() on all Append() calls
I20260926 20:42:59.843042 5761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:42:59.843787 5761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Bootstrap complete.
I20260926 20:42:59.856211 5761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } }
I20260926 20:42:59.857179 5761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e4b53c0952940df9755bc5595fab053, State: Initialized, Role: FOLLOWER
I20260926 20:42:59.857874 5761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } }
I20260926 20:42:59.858130 5761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:42:59.858285 5761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:42:59.858477 5761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 1 FOLLOWER]: Advancing to term 2
I20260926 20:42:59.863474 5761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } }
I20260926 20:42:59.864137 5761 leader_election.cc:302] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e4b53c0952940df9755bc5595fab053; no voters:
I20260926 20:42:59.864907 5761 leader_election.cc:288] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 20:42:59.865075 5765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 20:42:59.866253 5765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 2 LEADER]: Becoming Leader. State: Replica: 8e4b53c0952940df9755bc5595fab053, State: Running, Role: LEADER
I20260926 20:42:59.866963 5765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } }
I20260926 20:42:59.867441 5761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:42:59.869573 5767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e4b53c0952940df9755bc5595fab053. Latest consensus state: current_term: 2 leader_uuid: "8e4b53c0952940df9755bc5595fab053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } } }
I20260926 20:42:59.869975 5767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: This master's current role is: LEADER
I20260926 20:42:59.870725 5766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8e4b53c0952940df9755bc5595fab053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } } }
I20260926 20:42:59.871047 5766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: This master's current role is: LEADER
I20260926 20:42:59.881873 5770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:42:59.893190 5770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:42:59.895332 5770 catalog_manager.cc:1259] Loaded cluster ID: e76c028afacd4f968c57ec2efab64020
I20260926 20:42:59.895571 5770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:42:59.901160 5770 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 20:42:59.917073 5770 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 20:42:59.926838 5770 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:42:59.929687 5770 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Loaded TSK: 0
I20260926 20:42:59.931511 5770 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 20:43:00.206112 5763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:00.206418 5763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:00.206568 5763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:00.214222 5763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:00.214399 5763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:00.214587 5763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:43:00.224905 5763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:38609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=33601
--tserver_master_addrs=127.23.44.254:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:00.226321 5763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:00.227840 5763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:00.243211 5788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:00.243391 5789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:00.244144 5791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:00.244750 5763 server_base.cc:1034] running on GCE node
I20260926 20:43:00.245445 5763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:00.246511 5763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:00.247877 5763 hybrid_clock.cc:648] HybridClock initialized: now 1790455380247769 us; error 62 us; skew 500 ppm
I20260926 20:43:00.248490 5763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:00 dist-test-slave-4d5t krb5kdc[5140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455380, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:00.252418 5763 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:43:00.254194 5763 webserver.cc:460] Webserver started at http://127.23.44.193:33601/ using document root <none> and password file <none>
I20260926 20:43:00.254930 5763 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:00.255102 5763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:00.261359 5763 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 20:43:00.265165 5798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:00.266888 5763 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:43:00.267074 5763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "aad6447897c84bbe8edb1874bb20c162"
format_stamp: "Formatted at 2026-09-26 20:42:57 on dist-test-slave-4d5t"
server_key: "9779adf0f92a557f41af8e643f1b94d1"
server_key_iv: "740dd5990cef3485aabc135393f67c01"
server_key_version: "encryptionkey@0"
I20260926 20:43:00.268955 5763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:00.296121 5763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:00.300989 5763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:00.301277 5763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:00.302362 5763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:43:00.304227 5763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:43:00.304418 5763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:00.304674 5763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:43:00.304785 5763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:00.345252 5763 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:38609
I20260926 20:43:00.345429 5911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:38609 every 8 connection(s)
I20260926 20:43:00.346977 5763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:43:00.350165 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 5763
I20260926 20:43:00.351064 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:36159
--local_ip_for_outbound_sockets=127.23.44.194
--tserver_master_addrs=127.23.44.254:45935
--webserver_port=36191
--webserver_interface=127.23.44.194
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:43:00 dist-test-slave-4d5t krb5kdc[5140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455380, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:43:00.379725 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:00.350639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:39837 (local address 127.23.44.254:45935)
0926 20:43:00.351338 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:00.351356 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:00.353431 (+ 2075us) server_negotiation.cc:407] Connection header received
0926 20:43:00.357084 (+ 3653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:00.357097 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:00.357464 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:00.357770 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:00.359748 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:00.361333 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:00.364041 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:00.364822 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:00.364977 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:00.370275 (+ 5298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:00.370285 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:00.370303 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:00.370349 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:00.373918 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:00.375159 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:00.375166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:00.375172 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:00.375373 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:00.375945 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:00.375949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:00.375951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:00.376437 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:00.376646 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:00.377965 (+ 1319us) server_negotiation.cc:299] Negotiation successful
0926 20:43:00.378476 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":267,"threads_started":1}
I20260926 20:43:00.379849 5915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:00.351023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:39837)
0926 20:43:00.353221 (+ 2198us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:00.353281 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:00.356290 (+ 3009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:00.357993 (+ 1703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:00.358005 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:00.358835 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:43:00.359453 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:00.359471 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:00.361529 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:00.361538 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:00.363831 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:00.363853 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:00.365093 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:00.365102 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:00.365434 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:00.366284 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:43:00.366321 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:43:00.369880 (+ 3559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:43:00.374113 (+ 4233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:00.374128 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:00.374150 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:00.375001 (+ 851us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:00.375520 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:00.375525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:00.375531 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:00.375808 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:00.376626 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:43:00.376638 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:43:00.377127 (+ 489us) client_negotiation.cc:769] Sending connection context
0926 20:43:00.377513 (+ 386us) client_negotiation.cc:240] Negotiation successful
0926 20:43:00.378186 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1868,"thread_start_us":1609,"threads_started":1}
I20260926 20:43:00.383750 5912 heartbeater.cc:347] Connected to a master server at 127.23.44.254:45935
I20260926 20:43:00.384199 5912 heartbeater.cc:464] Registering TS with master...
I20260926 20:43:00.385406 5912 heartbeater.cc:511] Master 127.23.44.254:45935 requested a full tablet report, sending...
I20260926 20:43:00.388759 5726 ts_manager.cc:194] Registered new tserver with Master: aad6447897c84bbe8edb1874bb20c162 (127.23.44.193:38609)
I20260926 20:43:00.391522 5726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:39837
W20260926 20:43:00.717716 5916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:00.718011 5916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:00.718104 5916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:00.726366 5916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:00.726563 5916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:00.726750 5916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:43:00.736820 5916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:36159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=36191
--tserver_master_addrs=127.23.44.254:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:00.738178 5916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:00.739526 5916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:00.756379 5921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:00.756286 5922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:00.756348 5924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:00.757387 5916 server_base.cc:1034] running on GCE node
I20260926 20:43:00.758105 5916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:00.759145 5916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:00.760442 5916 hybrid_clock.cc:648] HybridClock initialized: now 1790455380760367 us; error 64 us; skew 500 ppm
I20260926 20:43:00.760833 5916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:00 dist-test-slave-4d5t krb5kdc[5140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455380, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:00.764166 5916 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:43:00.766317 5916 webserver.cc:460] Webserver started at http://127.23.44.194:36191/ using document root <none> and password file <none>
I20260926 20:43:00.767066 5916 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:00.767246 5916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:00.773690 5916 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 20:43:00.777150 5931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:00.778702 5916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:43:00.778843 5916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "5bd22737c91d42c498f6643804dd1d71"
format_stamp: "Formatted at 2026-09-26 20:42:57 on dist-test-slave-4d5t"
server_key: "0e36ac92fdf836fe701abefd08e5de6d"
server_key_iv: "8fee978216874a591913cff4bd7f84d3"
server_key_version: "encryptionkey@0"
I20260926 20:43:00.779872 5916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:00.798842 5916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:00.803668 5916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:00.803960 5916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:00.805079 5916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:43:00.806454 5916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:43:00.806591 5916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:00.806725 5916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:43:00.806823 5916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:00.846549 5916 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:36159
I20260926 20:43:00.846596 6043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:36159 every 8 connection(s)
I20260926 20:43:00.848438 5916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:43:00.851799 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 5916
I20260926 20:43:00.852519 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:38619
--local_ip_for_outbound_sockets=127.23.44.195
--tserver_master_addrs=127.23.44.254:45935
--webserver_port=33429
--webserver_interface=127.23.44.195
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:43:00 dist-test-slave-4d5t krb5kdc[5140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455380, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:43:00.874895 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:00.851508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:35271 (local address 127.23.44.254:45935)
0926 20:43:00.851730 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:00.851737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:00.853174 (+ 1437us) server_negotiation.cc:407] Connection header received
0926 20:43:00.856159 (+ 2985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:00.856165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:00.856252 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:00.856364 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:00.858653 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:00.859413 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:00.861564 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:00.862344 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:00.862500 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:00.866583 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:00.866593 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:00.866600 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:00.866651 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:00.870402 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:00.871623 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:00.871632 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:00.871638 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:00.871811 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:00.872636 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:00.872644 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:00.872649 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:00.873104 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:00.873302 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:00.874097 (+ 795us) server_negotiation.cc:299] Negotiation successful
0926 20:43:00.874405 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:43:00.876164 6046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:00.852034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:35271)
0926 20:43:00.852982 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:00.853037 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:00.855786 (+ 2749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:00.856683 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:00.856700 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:00.857552 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:43:00.858409 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:00.858437 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:00.859592 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:00.859601 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:00.861371 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:00.861384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:00.862586 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:00.862591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:00.862812 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:00.863570 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:43:00.863594 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:43:00.866304 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:43:00.870690 (+ 4386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:00.870702 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:00.870723 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:00.871418 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:00.872004 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:00.872009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:00.872012 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:00.872420 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:00.873263 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:43:00.873270 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:43:00.873662 (+ 392us) client_negotiation.cc:769] Sending connection context
0926 20:43:00.873970 (+ 308us) client_negotiation.cc:240] Negotiation successful
0926 20:43:00.874529 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":340,"threads_started":1}
I20260926 20:43:00.878509 6044 heartbeater.cc:347] Connected to a master server at 127.23.44.254:45935
I20260926 20:43:00.879040 6044 heartbeater.cc:464] Registering TS with master...
I20260926 20:43:00.879914 6044 heartbeater.cc:511] Master 127.23.44.254:45935 requested a full tablet report, sending...
I20260926 20:43:00.881695 5726 ts_manager.cc:194] Registered new tserver with Master: 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159)
I20260926 20:43:00.882697 5726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:35271
W20260926 20:43:01.203109 6047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:01.203390 6047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:01.203531 6047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:01.212015 6047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:01.212136 6047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:01.212322 6047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:43:01.223809 6047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:38619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=33429
--tserver_master_addrs=127.23.44.254:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:01.225281 6047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:01.226660 6047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:01.240425 6055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:01.240504 6052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:01.240463 6053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:01.246544 6047 server_base.cc:1034] running on GCE node
I20260926 20:43:01.247289 6047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:01.248394 6047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:01.249626 6047 hybrid_clock.cc:648] HybridClock initialized: now 1790455381249595 us; error 113 us; skew 500 ppm
I20260926 20:43:01.250036 6047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:01 dist-test-slave-4d5t krb5kdc[5140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455381, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:01.253713 6047 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:43:01.255461 6047 webserver.cc:460] Webserver started at http://127.23.44.195:33429/ using document root <none> and password file <none>
I20260926 20:43:01.256171 6047 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:01.256285 6047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:01.262518 6047 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:43:01.266215 6062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:01.268290 6047 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:43:01.268543 6047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "387b20d842b6457b9ab32a65fd6c447c"
format_stamp: "Formatted at 2026-09-26 20:42:58 on dist-test-slave-4d5t"
server_key: "85d5a20e5a8218e834f4941a86d125fc"
server_key_iv: "3f31d2db53973ec96debc462d5d10936"
server_key_version: "encryptionkey@0"
I20260926 20:43:01.269743 6047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:01.310797 6047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:01.315727 6047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:01.316047 6047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:01.317090 6047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:43:01.318465 6047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:43:01.318598 6047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:01.318737 6047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:43:01.318830 6047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:01.357553 6047 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:38619
I20260926 20:43:01.357647 6174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:38619 every 8 connection(s)
I20260926 20:43:01.359058 6047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:43:01.362607 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 6047
Sep 26 20:43:01 dist-test-slave-4d5t krb5kdc[5140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455381, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:43:01.380785 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.363564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50470 (local address 127.23.44.254:45935)
0926 20:43:01.364077 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:01.364084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:01.364460 (+ 376us) server_negotiation.cc:407] Connection header received
0926 20:43:01.364695 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:01.364701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:01.364786 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:01.364920 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:01.366438 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.367565 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.369328 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.369947 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.370076 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.373459 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:01.373466 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:01.373472 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:01.373511 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:01.377633 (+ 4122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:01.378495 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:01.378501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:01.378505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:01.378672 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:01.379109 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:01.379115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:01.379118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:01.379444 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:01.379719 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:01.380058 (+ 339us) server_negotiation.cc:299] Negotiation successful
0926 20:43:01.380317 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":243,"threads_started":1}
I20260926 20:43:01.381889 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.361926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:38785 (local address 127.23.44.254:45935)
0926 20:43:01.362176 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:01.362187 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:01.363375 (+ 1188us) server_negotiation.cc:407] Connection header received
0926 20:43:01.365809 (+ 2434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:01.365814 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:01.365892 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:01.366009 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:01.368013 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.368799 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.371026 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.371483 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.371619 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.375782 (+ 4163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:01.375787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:01.375790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:01.375833 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:01.378659 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:01.379455 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:01.379462 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:01.379465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:01.379584 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:01.380033 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:01.380039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:01.380042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:01.380299 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:01.380449 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:01.381342 (+ 893us) server_negotiation.cc:299] Negotiation successful
0926 20:43:01.381548 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:43:01.383087 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.362371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:38785)
0926 20:43:01.363216 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:01.363263 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:01.365557 (+ 2294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:01.366268 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:01.366277 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:01.367018 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:43:01.367843 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:01.367863 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.368985 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.368991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:01.370865 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:01.370890 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.371723 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.371731 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:01.371970 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.372661 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:43:01.372685 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:43:01.375542 (+ 2857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:43:01.378822 (+ 3280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:01.378832 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:01.378851 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:01.379303 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:01.379700 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:01.379705 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:01.379709 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:01.379912 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:01.380437 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:43:01.380445 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:43:01.380894 (+ 449us) client_negotiation.cc:769] Sending connection context
0926 20:43:01.381263 (+ 369us) client_negotiation.cc:240] Negotiation successful
0926 20:43:01.381838 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":360,"threads_started":1}
I20260926 20:43:01.384706 6175 heartbeater.cc:347] Connected to a master server at 127.23.44.254:45935
I20260926 20:43:01.384999 6175 heartbeater.cc:464] Registering TS with master...
I20260926 20:43:01.385705 6175 heartbeater.cc:511] Master 127.23.44.254:45935 requested a full tablet report, sending...
I20260926 20:43:01.387796 5726 ts_manager.cc:194] Registered new tserver with Master: 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619)
I20260926 20:43:01.389010 5726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:38785
I20260926 20:43:01.396191 5912 heartbeater.cc:503] Master 127.23.44.254:45935 was elected leader, sending a full tablet report...
I20260926 20:43:01.396255 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:43:01.403676 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.397339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50480 (local address 127.23.44.254:45935)
0926 20:43:01.397556 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:01.397566 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:01.398069 (+ 503us) server_negotiation.cc:407] Connection header received
0926 20:43:01.398228 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:01.398232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:01.398357 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:01.398475 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:43:01.398827 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.399679 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.401552 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.402000 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.402106 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.402628 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:43:01.403150 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:43:01.403263 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:01.403395 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 20:43:01.403470 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:43:01.404768 5726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:50480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:43:01.412227 5726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:43:01.438020 6187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.424746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:36159 (local address 127.0.0.1:60822)
0926 20:43:01.426624 (+ 1878us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:01.426641 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:01.426903 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:01.428093 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:01.428101 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:01.428354 (+ 253us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:01.428950 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:01.428964 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.431317 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.431323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:01.433791 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:01.433818 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.436730 (+ 2912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.436736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:01.436957 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.436984 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 20:43:01.437149 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 20:43:01.437435 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1757,"mutex_wait_us":188,"thread_start_us":1605,"threads_started":1}
I20260926 20:43:01.438112 6185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.424690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:38609 (local address 127.0.0.1:35226)
0926 20:43:01.425558 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:01.425601 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:01.425760 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:01.428093 (+ 2333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:01.428101 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:01.428429 (+ 328us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:01.428961 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:01.428970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.430612 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.430618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:01.433964 (+ 3346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:01.433983 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.436756 (+ 2773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.436763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:01.436936 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.436970 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 20:43:01.437138 (+ 168us) client_negotiation.cc:240] Negotiation successful
0926 20:43:01.437417 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":239,"spinlock_wait_cycles":584576,"thread_start_us":494,"threads_started":1}
I20260926 20:43:01.438248 6189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.427154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44254 (local address 127.23.44.195:38619)
0926 20:43:01.427740 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:01.427747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:01.428471 (+ 724us) server_negotiation.cc:407] Connection header received
0926 20:43:01.428797 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:01.428803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:01.428965 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:01.429146 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:01.430001 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.431566 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.434447 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.435989 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.436104 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.436357 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:01.437216 (+ 859us) server_negotiation.cc:299] Negotiation successful
0926 20:43:01.437416 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":755,"threads_started":1}
I20260926 20:43:01.438136 6190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.426557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:38619 (local address 127.0.0.1:44254)
0926 20:43:01.428355 (+ 1798us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:01.428380 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:01.428622 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:01.429420 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:01.429425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:01.429448 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:01.429802 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:01.429813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.431736 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.431742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:01.434244 (+ 2502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:01.434258 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.436622 (+ 2364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.436630 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:01.436860 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.436969 (+ 109us) client_negotiation.cc:769] Sending connection context
0926 20:43:01.437147 (+ 178us) client_negotiation.cc:240] Negotiation successful
0926 20:43:01.437417 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1620,"mutex_wait_us":108,"thread_start_us":1595,"threads_started":1}
I20260926 20:43:01.439144 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.425228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60822 (local address 127.23.44.194:36159)
0926 20:43:01.426160 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:01.426169 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:01.426728 (+ 559us) server_negotiation.cc:407] Connection header received
0926 20:43:01.427231 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:01.427240 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:01.427578 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:01.427795 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:01.429143 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.430983 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.434080 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.436147 (+ 2067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.436316 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.436536 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:01.438623 (+ 2087us) server_negotiation.cc:299] Negotiation successful
0926 20:43:01.438867 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":778,"thread_start_us":656,"threads_started":1}
I20260926 20:43:01.439945 6186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.424786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35226 (local address 127.23.44.193:38609)
0926 20:43:01.427211 (+ 2425us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:01.427218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:01.427242 (+ 24us) server_negotiation.cc:407] Connection header received
0926 20:43:01.427321 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:01.427327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:01.427599 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:01.427795 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:01.429188 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.430429 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.434183 (+ 3754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.435989 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.436118 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.436574 (+ 456us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:01.439436 (+ 2862us) server_negotiation.cc:299] Negotiation successful
0926 20:43:01.439581 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2271,"thread_start_us":2098,"threads_started":1}
I20260926 20:43:01.442494 6110 tablet_service.cc:1467] Processing CreateTablet for tablet f81a5bf9e9f1405bb9fdf1c9745b6fe6 (DEFAULT_TABLE table=test-table [id=74a440208b2b4619a571bb6f74137c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:43:01.442624 5979 tablet_service.cc:1467] Processing CreateTablet for tablet f81a5bf9e9f1405bb9fdf1c9745b6fe6 (DEFAULT_TABLE table=test-table [id=74a440208b2b4619a571bb6f74137c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:43:01.442865 5847 tablet_service.cc:1467] Processing CreateTablet for tablet f81a5bf9e9f1405bb9fdf1c9745b6fe6 (DEFAULT_TABLE table=test-table [id=74a440208b2b4619a571bb6f74137c21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:43:01.443996 6110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f81a5bf9e9f1405bb9fdf1c9745b6fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:43:01.444010 5979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f81a5bf9e9f1405bb9fdf1c9745b6fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:43:01.444206 5847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f81a5bf9e9f1405bb9fdf1c9745b6fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:43:01.459900 6191 tablet_bootstrap.cc:492] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: Bootstrap starting.
I20260926 20:43:01.464794 6191 tablet_bootstrap.cc:654] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: Neither blocks nor log segments found. Creating new log.
I20260926 20:43:01.465938 6191 log.cc:824] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: Log is configured to *not* fsync() on all Append() calls
I20260926 20:43:01.466166 6192 tablet_bootstrap.cc:492] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71: Bootstrap starting.
I20260926 20:43:01.467519 6193 tablet_bootstrap.cc:492] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c: Bootstrap starting.
I20260926 20:43:01.469826 6191 tablet_bootstrap.cc:492] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: No bootstrap required, opened a new log
I20260926 20:43:01.470168 6191 ts_tablet_manager.cc:1397] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260926 20:43:01.470949 6193 tablet_bootstrap.cc:654] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c: Neither blocks nor log segments found. Creating new log.
I20260926 20:43:01.471103 6192 tablet_bootstrap.cc:654] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71: Neither blocks nor log segments found. Creating new log.
I20260926 20:43:01.472114 6193 log.cc:824] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c: Log is configured to *not* fsync() on all Append() calls
I20260926 20:43:01.472355 6192 log.cc:824] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71: Log is configured to *not* fsync() on all Append() calls
I20260926 20:43:01.475346 6193 tablet_bootstrap.cc:492] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c: No bootstrap required, opened a new log
I20260926 20:43:01.475618 6193 ts_tablet_manager.cc:1397] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260926 20:43:01.479310 6191 raft_consensus.cc:348] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:01.479718 6191 raft_consensus.cc:374] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:43:01.479821 6191 raft_consensus.cc:729] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad6447897c84bbe8edb1874bb20c162, State: Initialized, Role: FOLLOWER
I20260926 20:43:01.480526 6191 consensus_queue.cc:260] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:01.482458 6192 tablet_bootstrap.cc:492] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71: No bootstrap required, opened a new log
I20260926 20:43:01.482860 6192 ts_tablet_manager.cc:1397] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.000s
I20260926 20:43:01.484139 6191 ts_tablet_manager.cc:1428] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260926 20:43:01.483940 6193 raft_consensus.cc:348] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:01.484320 6193 raft_consensus.cc:374] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:43:01.484431 6193 raft_consensus.cc:729] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 387b20d842b6457b9ab32a65fd6c447c, State: Initialized, Role: FOLLOWER
I20260926 20:43:01.485183 6193 consensus_queue.cc:260] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:01.486347 6175 heartbeater.cc:503] Master 127.23.44.254:45935 was elected leader, sending a full tablet report...
I20260926 20:43:01.487010 6193 ts_tablet_manager.cc:1428] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c: Time spent starting tablet: real 0.011s user 0.012s sys 0.001s
I20260926 20:43:01.495736 6192 raft_consensus.cc:348] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:01.496266 6192 raft_consensus.cc:374] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:43:01.496424 6192 raft_consensus.cc:729] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bd22737c91d42c498f6643804dd1d71, State: Initialized, Role: FOLLOWER
I20260926 20:43:01.497195 6192 consensus_queue.cc:260] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:01.498886 6044 heartbeater.cc:503] Master 127.23.44.254:45935 was elected leader, sending a full tablet report...
I20260926 20:43:01.499651 6192 ts_tablet_manager.cc:1428] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71: Time spent starting tablet: real 0.017s user 0.010s sys 0.004s
W20260926 20:43:01.600237 5913 tablet.cc:2369] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:43:01.600917 6045 tablet.cc:2369] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:43:01.610864 6176 tablet.cc:2369] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:43:01.697669 6198 raft_consensus.cc:482] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:43:01.698164 6198 raft_consensus.cc:504] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:01.700258 6198 leader_election.cc:288] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aad6447897c84bbe8edb1874bb20c162 (127.23.44.193:38609), 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159)
I20260926 20:43:01.709139 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.701020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:49907 (local address 127.23.44.194:36159)
0926 20:43:01.701540 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:01.701547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:01.702386 (+ 839us) server_negotiation.cc:407] Connection header received
0926 20:43:01.702552 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:01.702556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:01.702620 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:01.702705 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:01.703534 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.704446 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.706792 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.708070 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.708198 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.708248 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:01.708774 (+ 526us) server_negotiation.cc:299] Negotiation successful
0926 20:43:01.708903 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260926 20:43:01.709139 6186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.700751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:46105 (local address 127.23.44.193:38609)
0926 20:43:01.700980 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:01.700986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:01.701573 (+ 587us) server_negotiation.cc:407] Connection header received
0926 20:43:01.701924 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:01.701931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:01.702029 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:01.702166 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:01.702867 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.703831 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.706432 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.707679 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.707886 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.707975 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:01.708677 (+ 702us) server_negotiation.cc:299] Negotiation successful
0926 20:43:01.708846 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 20:43:01.709334 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.700762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:38609 (local address 127.23.44.195:46105)
0926 20:43:01.701171 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:01.701387 (+ 216us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:01.701718 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:01.702237 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:01.702243 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:01.702279 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:01.702644 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:01.702654 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.704050 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.704057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:01.706206 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:01.706229 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.707934 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.707959 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:01.708328 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.708417 (+ 89us) client_negotiation.cc:769] Sending connection context
0926 20:43:01.708613 (+ 196us) client_negotiation.cc:240] Negotiation successful
0926 20:43:01.708812 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":33,"spinlock_wait_cycles":35200}
I20260926 20:43:01.709895 6200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.700839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:36159 (local address 127.23.44.195:49907)
0926 20:43:01.702243 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:01.702265 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:01.702417 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:01.702763 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:01.702771 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:01.702808 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:01.703311 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:01.703326 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.704647 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.704654 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:01.706628 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:01.706638 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.708333 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.708339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:01.708603 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.708636 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 20:43:01.708727 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 20:43:01.708855 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1220,"mutex_wait_us":4,"thread_start_us":958,"threads_started":1}
I20260926 20:43:01.710969 5867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f81a5bf9e9f1405bb9fdf1c9745b6fe6" candidate_uuid: "387b20d842b6457b9ab32a65fd6c447c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aad6447897c84bbe8edb1874bb20c162" is_pre_election: true
I20260926 20:43:01.711510 5999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f81a5bf9e9f1405bb9fdf1c9745b6fe6" candidate_uuid: "387b20d842b6457b9ab32a65fd6c447c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bd22737c91d42c498f6643804dd1d71" is_pre_election: true
I20260926 20:43:01.711833 5867 raft_consensus.cc:2454] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 387b20d842b6457b9ab32a65fd6c447c in term 0.
I20260926 20:43:01.712035 5999 raft_consensus.cc:2454] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 387b20d842b6457b9ab32a65fd6c447c in term 0.
I20260926 20:43:01.712776 6064 leader_election.cc:302] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 387b20d842b6457b9ab32a65fd6c447c, 5bd22737c91d42c498f6643804dd1d71, aad6447897c84bbe8edb1874bb20c162; no voters:
I20260926 20:43:01.713244 6198 raft_consensus.cc:2781] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:43:01.713424 6198 raft_consensus.cc:482] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:43:01.713548 6198 raft_consensus.cc:3037] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:43:01.715972 6198 raft_consensus.cc:504] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:01.716804 6198 leader_election.cc:288] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [CANDIDATE]: Term 1 election: Requested vote from peers aad6447897c84bbe8edb1874bb20c162 (127.23.44.193:38609), 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159)
I20260926 20:43:01.717263 5867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f81a5bf9e9f1405bb9fdf1c9745b6fe6" candidate_uuid: "387b20d842b6457b9ab32a65fd6c447c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aad6447897c84bbe8edb1874bb20c162"
I20260926 20:43:01.717267 5999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f81a5bf9e9f1405bb9fdf1c9745b6fe6" candidate_uuid: "387b20d842b6457b9ab32a65fd6c447c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bd22737c91d42c498f6643804dd1d71"
I20260926 20:43:01.717509 5999 raft_consensus.cc:3037] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:43:01.717502 5867 raft_consensus.cc:3037] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:43:01.720847 5999 raft_consensus.cc:2454] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 387b20d842b6457b9ab32a65fd6c447c in term 1.
I20260926 20:43:01.720847 5867 raft_consensus.cc:2454] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 387b20d842b6457b9ab32a65fd6c447c in term 1.
I20260926 20:43:01.721470 6065 leader_election.cc:302] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 387b20d842b6457b9ab32a65fd6c447c, 5bd22737c91d42c498f6643804dd1d71, aad6447897c84bbe8edb1874bb20c162; no voters:
I20260926 20:43:01.721886 6198 raft_consensus.cc:2781] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:43:01.722375 6198 raft_consensus.cc:686] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [term 1 LEADER]: Becoming Leader. State: Replica: 387b20d842b6457b9ab32a65fd6c447c, State: Running, Role: LEADER
I20260926 20:43:01.722888 6198 consensus_queue.cc:237] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:01.728286 5725 catalog_manager.cc:5495] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c reported cstate change: term changed from 0 to 1, leader changed from <none> to 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195). New cstate: current_term: 1 leader_uuid: "387b20d842b6457b9ab32a65fd6c447c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } health_report { overall_health: HEALTHY } } }
I20260926 20:43:01.814954 6189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.809389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44266 (local address 127.23.44.195:38619)
0926 20:43:01.809603 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:01.809609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:01.809715 (+ 106us) server_negotiation.cc:407] Connection header received
0926 20:43:01.809970 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:01.809975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:01.810063 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:01.810226 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:43:01.810592 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.811301 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.812765 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.813179 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.813317 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.813824 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:43:01.814348 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:43:01.814510 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:01.814591 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 20:43:01.814675 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:43:01.823848 5999 raft_consensus.cc:1261] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71 [term 1 FOLLOWER]: Refusing update from remote peer 387b20d842b6457b9ab32a65fd6c447c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:43:01.823874 5867 raft_consensus.cc:1261] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Refusing update from remote peer 387b20d842b6457b9ab32a65fd6c447c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:43:01.824954 6201 consensus_queue.cc:1035] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:43:01.825330 6198 consensus_queue.cc:1035] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:43:01.838778 6205 mvcc.cc:205] Tried to move back new op lower bound from 7333705243939876864 to 7333705243540697088. Current Snapshot: MvccSnapshot[applied={T|T < 7333705243939876864}]
I20260926 20:43:01.842794 6206 mvcc.cc:205] Tried to move back new op lower bound from 7333705243939876864 to 7333705243540697088. Current Snapshot: MvccSnapshot[applied={T|T < 7333705243939876864}]
I20260926 20:43:01.870656 5726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:50480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:43:01.871007 5726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:50480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:43:01.875255 5726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Sending DeleteTablet for 3 replicas of tablet f81a5bf9e9f1405bb9fdf1c9745b6fe6
I20260926 20:43:01.876971 6110 tablet_service.cc:1514] Processing DeleteTablet for tablet f81a5bf9e9f1405bb9fdf1c9745b6fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:43:01 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:44254
I20260926 20:43:01.877005 5847 tablet_service.cc:1514] Processing DeleteTablet for tablet f81a5bf9e9f1405bb9fdf1c9745b6fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:43:01 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:35226
I20260926 20:43:01.877930 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 5763
I20260926 20:43:01.878186 5979 tablet_service.cc:1514] Processing DeleteTablet for tablet f81a5bf9e9f1405bb9fdf1c9745b6fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:43:01 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:60822
I20260926 20:43:01.902215 6215 tablet_replica.cc:330] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: stopping tablet replica
I20260926 20:43:01.902904 6216 tablet_replica.cc:330] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71: stopping tablet replica
I20260926 20:43:01.903553 6186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:01.879621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35228 (local address 127.23.44.193:38609)
0926 20:43:01.879834 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:01.879842 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:01.879864 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:43:01.879933 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:01.879942 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:01.880039 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:01.880148 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:01.882532 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.883954 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.886539 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:01.887490 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:01.887655 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:01.892931 (+ 5276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:01.892959 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:01.892986 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:01.893060 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:01.898216 (+ 5156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:01.899612 (+ 1396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:01.899625 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:01.899630 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:01.899847 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:01.900395 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:01.900402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:01.900408 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:01.901121 (+ 713us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:01.901501 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:01.902905 (+ 1404us) server_negotiation.cc:299] Negotiation successful
0926 20:43:01.903181 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:43:01.903550 6216 raft_consensus.cc:2229] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:43:01.904315 5907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:01.905391 6214 tablet_replica.cc:330] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c: stopping tablet replica
I20260926 20:43:01.905911 6214 raft_consensus.cc:2229] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:43:01.906788 6214 raft_consensus.cc:2258] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:43:01.907486 6216 raft_consensus.cc:2258] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:43:01.909880 6214 ts_tablet_manager.cc:1905] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:43:01.909991 6216 ts_tablet_manager.cc:1905] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:43:01.918290 6214 ts_tablet_manager.cc:1918] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:43:01.918605 6214 log.cc:1196] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/f81a5bf9e9f1405bb9fdf1c9745b6fe6
I20260926 20:43:01.919406 6214 ts_tablet_manager.cc:1939] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 387b20d842b6457b9ab32a65fd6c447c: Deleting consensus metadata
I20260926 20:43:01.920109 6216 ts_tablet_manager.cc:1918] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:43:01.920341 6216 log.cc:1196] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/f81a5bf9e9f1405bb9fdf1c9745b6fe6
I20260926 20:43:01.921070 6216 ts_tablet_manager.cc:1939] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P 5bd22737c91d42c498f6643804dd1d71: Deleting consensus metadata
I20260926 20:43:01.921847 5710 catalog_manager.cc:4841] TS 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619): tablet f81a5bf9e9f1405bb9fdf1c9745b6fe6 (table test-table [id=74a440208b2b4619a571bb6f74137c21]) successfully deleted
I20260926 20:43:01.923339 5711 catalog_manager.cc:4841] TS 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159): tablet f81a5bf9e9f1405bb9fdf1c9745b6fe6 (table test-table [id=74a440208b2b4619a571bb6f74137c21]) successfully deleted
I20260926 20:43:01.904668 6215 raft_consensus.cc:2229] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:43:02.039489 6215 raft_consensus.cc:2258] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:43:02.039880 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 5763
W20260926 20:43:02.061604 5712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.44.193:38609 (error 108)
I20260926 20:43:02.061834 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 5916
I20260926 20:43:02.064294 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:02.063520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:38609 (local address 127.0.0.1:35242)
0926 20:43:02.063954 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:02.064078 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:38609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":235,"threads_started":1}
W20260926 20:43:02.064693 5712 catalog_manager.cc:4584] TS aad6447897c84bbe8edb1874bb20c162 (127.23.44.193:38609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f81a5bf9e9f1405bb9fdf1c9745b6fe6: Network error: Client connection negotiation failed: client connection to 127.23.44.193:38609: connect: Connection refused (error 111)
I20260926 20:43:02.081063 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:02.062726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60826 (local address 127.23.44.194:36159)
0926 20:43:02.063135 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:02.063143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:02.063184 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:43:02.063412 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:02.063420 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:02.063619 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:02.063782 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:02.065695 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:02.066815 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:02.068355 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:02.069080 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:02.069212 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:02.072039 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:02.072058 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:02.072078 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:02.072125 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:02.076051 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:02.078069 (+ 2018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:02.078096 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:02.078102 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:02.078327 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:02.078916 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:02.078924 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:02.078927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:02.079636 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:02.079981 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:02.080355 (+ 374us) server_negotiation.cc:299] Negotiation successful
0926 20:43:02.080636 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 20:43:02.081805 6039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:02.118366 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:02.117808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:38609 (local address 127.0.0.1:35256)
0926 20:43:02.118057 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:02.118186 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:38609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260926 20:43:02.119549 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:02.119126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:38609 (local address 127.0.0.1:35264)
0926 20:43:02.119286 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:02.119373 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:38609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 20:43:02.180955 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:02.180204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:38609 (local address 127.0.0.1:35274)
0926 20:43:02.180586 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:02.180737 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:38609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260926 20:43:02.182624 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:02.182160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:38609 (local address 127.0.0.1:35280)
0926 20:43:02.182325 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:02.182421 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:38609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 20:43:02.190994 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 5916
I20260926 20:43:02.209477 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 6047
I20260926 20:43:02.226501 6189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:02.210256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44268 (local address 127.23.44.195:38619)
0926 20:43:02.210448 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:02.210455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:02.210562 (+ 107us) server_negotiation.cc:407] Connection header received
0926 20:43:02.210766 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:02.210771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:02.210858 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:02.210970 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:02.213304 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:02.214378 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:02.215934 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:02.216455 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:02.216736 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:02.219575 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:02.219591 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:02.219608 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:02.219655 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:02.223151 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:02.223954 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:02.223963 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:02.223969 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:02.224144 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:02.224726 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:02.224735 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:02.224741 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:02.225135 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:02.225538 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:02.225807 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 20:43:02.226065 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 20:43:02.227321 6170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:02.263733 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:02.263187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:38609 (local address 127.0.0.1:35288)
0926 20:43:02.263468 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:02.263561 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:38609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260926 20:43:02.264984 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:02.264457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:38609 (local address 127.0.0.1:35298)
0926 20:43:02.264745 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:02.264841 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:38609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260926 20:43:02.358732 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 6047
I20260926 20:43:02.376628 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 5693
I20260926 20:43:02.378082 5756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:02.500540 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:02.499874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:38609 (local address 127.0.0.1:35308)
0926 20:43:02.500141 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:02.500318 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:38609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 20:43:02.500543 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 5693
I20260926 20:43:02.523413 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:45935
--webserver_interface=127.23.44.254
--webserver_port=38347
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:45935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:43:02.874615 6223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:02.874950 6223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:02.875053 6223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:02.883708 6223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:43:02.883829 6223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:02.883877 6223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:02.883930 6223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:43:02.883977 6223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:43:02.894600 6223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:45935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:45935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=38347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:02.895958 6223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:02.897389 6223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:02.909890 6231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:02.909941 6229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:02.910166 6228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:02.911636 6223 server_base.cc:1034] running on GCE node
I20260926 20:43:02.912501 6223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:02.914312 6223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:02.915563 6223 hybrid_clock.cc:648] HybridClock initialized: now 1790455382915523 us; error 38 us; skew 500 ppm
I20260926 20:43:02.916000 6223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:02 dist-test-slave-4d5t krb5kdc[5140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455382, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:02.919533 6223 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:43:02.921352 6223 webserver.cc:460] Webserver started at http://127.23.44.254:38347/ using document root <none> and password file <none>
I20260926 20:43:02.922096 6223 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:02.922204 6223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:02.928119 6223 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 20:43:02.933964 6238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:02.935662 6223 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:43:02.935845 6223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "8e4b53c0952940df9755bc5595fab053"
format_stamp: "Formatted at 2026-09-26 20:42:56 on dist-test-slave-4d5t"
server_key: "739fc0ed7b955be7403681409957d2e5"
server_key_iv: "d9b05c638c61c746ba2a644969ef994c"
server_key_version: "encryptionkey@0"
I20260926 20:43:02.937000 6223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:02.959299 6223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:02.964264 6223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:02.964659 6223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:02.986891 6223 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:45935
I20260926 20:43:02.986920 6289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:45935 every 8 connection(s)
I20260926 20:43:02.989095 6223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:43:02.991587 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 6223
I20260926 20:43:02.992714 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:38609
--local_ip_for_outbound_sockets=127.23.44.193
--tserver_master_addrs=127.23.44.254:45935
--webserver_port=33601
--webserver_interface=127.23.44.193
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:43:03.000531 6290 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:43:03.005482 6290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Bootstrap starting.
I20260926 20:43:03.027308 6290 log.cc:824] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Log is configured to *not* fsync() on all Append() calls
I20260926 20:43:03.044771 6290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:43:03.045387 6290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053: Bootstrap complete.
I20260926 20:43:03.058625 6290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } }
I20260926 20:43:03.059629 6290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e4b53c0952940df9755bc5595fab053, State: Initialized, Role: FOLLOWER
I20260926 20:43:03.060279 6290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } }
I20260926 20:43:03.060540 6290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:43:03.060683 6290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:43:03.060874 6290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 2 FOLLOWER]: Advancing to term 3
I20260926 20:43:03.066418 6290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } }
I20260926 20:43:03.067236 6290 leader_election.cc:302] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e4b53c0952940df9755bc5595fab053; no voters:
I20260926 20:43:03.068100 6290 leader_election.cc:288] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 20:43:03.068457 6294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 20:43:03.069947 6294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [term 3 LEADER]: Becoming Leader. State: Replica: 8e4b53c0952940df9755bc5595fab053, State: Running, Role: LEADER
I20260926 20:43:03.070596 6294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } }
I20260926 20:43:03.070941 6290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:43:03.073295 6296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e4b53c0952940df9755bc5595fab053. Latest consensus state: current_term: 3 leader_uuid: "8e4b53c0952940df9755bc5595fab053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } } }
I20260926 20:43:03.073734 6296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: This master's current role is: LEADER
I20260926 20:43:03.074375 6295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8e4b53c0952940df9755bc5595fab053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e4b53c0952940df9755bc5595fab053" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45935 } } }
I20260926 20:43:03.074693 6295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e4b53c0952940df9755bc5595fab053 [sys.catalog]: This master's current role is: LEADER
I20260926 20:43:03.084928 6300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:43:03.095535 6300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:43:03.100639 6300 catalog_manager.cc:1259] Loaded cluster ID: e76c028afacd4f968c57ec2efab64020
I20260926 20:43:03.100828 6300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 20:43:03.104157 6300 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f148b4139ad google::LogMessage::Fail() at ??:0
@ 0x7f148b41688a google::LogMessage::SendToLog() at ??:0
@ 0x7f148b4134de google::LogMessage::Flush() at ??:0
@ 0x7f148b414a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260926 20:43:03.123426 6312 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f14a0e4a512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f14a0e47976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f14a0ecf610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f14a0ecf439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f148e05f345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f148e06291c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f148e062799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f148e03deaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f14913176db start_thread at ??:0
@ 0x7f148a2ad61f clone at ??:0
W20260926 20:43:03.402697 6292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:03.403033 6292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:03.403204 6292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:03.410808 6292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:03.410979 6292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:03.411155 6292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:43:03.422003 6292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:38609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=33601
--tserver_master_addrs=127.23.44.254:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:03.424108 6292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:03.425905 6292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:03.439913 6331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:03.439816 6332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:03.439829 6334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:03.441471 6292 server_base.cc:1034] running on GCE node
I20260926 20:43:03.442374 6292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:03.443467 6292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:03.444813 6292 hybrid_clock.cc:648] HybridClock initialized: now 1790455383444756 us; error 70 us; skew 500 ppm
I20260926 20:43:03.445472 6292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:03 dist-test-slave-4d5t krb5kdc[5140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455383, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:03.449254 6292 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:43:03.450918 6292 webserver.cc:460] Webserver started at http://127.23.44.193:33601/ using document root <none> and password file <none>
I20260926 20:43:03.451653 6292 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:03.451812 6292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:03.457489 6292 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260926 20:43:03.460690 6341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:03.462527 6292 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260926 20:43:03.462749 6292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "aad6447897c84bbe8edb1874bb20c162"
format_stamp: "Formatted at 2026-09-26 20:42:57 on dist-test-slave-4d5t"
server_key: "9779adf0f92a557f41af8e643f1b94d1"
server_key_iv: "740dd5990cef3485aabc135393f67c01"
server_key_version: "encryptionkey@0"
I20260926 20:43:03.463857 6292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:03.488543 6292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:03.493525 6292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:03.493849 6292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:03.494910 6292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:43:03.497018 6348 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260926 20:43:03.504200 6292 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 20:43:03.504426 6292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.000s sys 0.001s
I20260926 20:43:03.504700 6292 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 20:43:03.508528 6292 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 20:43:03.508700 6292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.000s sys 0.004s
I20260926 20:43:03.509122 6348 tablet_bootstrap.cc:492] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: Bootstrap starting.
I20260926 20:43:03.539129 6348 log.cc:824] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: Log is configured to *not* fsync() on all Append() calls
I20260926 20:43:03.552711 6292 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:38609
I20260926 20:43:03.552742 6455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:38609 every 8 connection(s)
I20260926 20:43:03.554155 6292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:43:03.555040 6348 tablet_bootstrap.cc:492] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:43:03.555714 6348 tablet_bootstrap.cc:492] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: Bootstrap complete.
I20260926 20:43:03.555945 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 6292
I20260926 20:43:03.557207 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:36159
--local_ip_for_outbound_sockets=127.23.44.194
--tserver_master_addrs=127.23.44.254:45935
--webserver_port=36191
--webserver_interface=127.23.44.194
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:43:03.557263 6348 ts_tablet_manager.cc:1397] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: Time spent bootstrapping tablet: real 0.048s user 0.021s sys 0.025s
I20260926 20:43:03.569326 6458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:03.557260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:41963)
0926 20:43:03.567564 (+ 10304us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:03.567896 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9870,"threads_started":1}
W20260926 20:43:03.571306 6456 heartbeater.cc:644] Failed to heartbeat to 127.23.44.254:45935 (0 consecutive failures): Network error: Failed to ping master at 127.23.44.254:45935: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
I20260926 20:43:03.570492 6348 raft_consensus.cc:348] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:03.571900 6348 raft_consensus.cc:729] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad6447897c84bbe8edb1874bb20c162, State: Initialized, Role: FOLLOWER
I20260926 20:43:03.572510 6458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:03.571954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:48811)
0926 20:43:03.572268 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:03.572354 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260926 20:43:03.572633 6348 consensus_queue.cc:260] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:03.575379 6458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:03.573105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:33627)
0926 20:43:03.574991 (+ 1886us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:03.575107 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1737,"threads_started":1}
W20260926 20:43:03.577148 6456 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 20:43:03.578346 6348 ts_tablet_manager.cc:1428] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162: Time spent starting tablet: real 0.021s user 0.008s sys 0.012s
I20260926 20:43:03.578684 6461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:03.578044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:45249)
0926 20:43:03.578254 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:03.578485 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260926 20:43:03.579527 6461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:03.579209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:35393)
0926 20:43:03.579355 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:03.579427 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 20:43:03.908681 6459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:03.909047 6459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:03.909224 6459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:03.916882 6459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:03.917048 6459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:03.917239 6459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:43:03.929423 6459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:36159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=36191
--tserver_master_addrs=127.23.44.254:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:03.930966 6459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:03.932314 6459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:03.946345 6469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:03.946518 6466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:03.946867 6467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:03.948251 6459 server_base.cc:1034] running on GCE node
I20260926 20:43:03.949347 6459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:03.950809 6459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:03.952126 6459 hybrid_clock.cc:648] HybridClock initialized: now 1790455383952064 us; error 62 us; skew 500 ppm
I20260926 20:43:03.952623 6459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:03 dist-test-slave-4d5t krb5kdc[5140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455383, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:03.956336 6459 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:43:03.958242 6459 webserver.cc:460] Webserver started at http://127.23.44.194:36191/ using document root <none> and password file <none>
I20260926 20:43:03.959022 6459 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:03.959189 6459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:03.965696 6459 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 20:43:03.969899 6476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:03.971793 6459 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260926 20:43:03.972016 6459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "5bd22737c91d42c498f6643804dd1d71"
format_stamp: "Formatted at 2026-09-26 20:42:57 on dist-test-slave-4d5t"
server_key: "0e36ac92fdf836fe701abefd08e5de6d"
server_key_iv: "8fee978216874a591913cff4bd7f84d3"
server_key_version: "encryptionkey@0"
I20260926 20:43:03.973212 6459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:03.992535 6459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:03.997116 6459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:03.997427 6459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:03.998488 6459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:43:04.000099 6459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:43:04.000217 6459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:04.000353 6459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:43:04.000458 6459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:04.039531 6459 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:36159
I20260926 20:43:04.039583 6588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:36159 every 8 connection(s)
I20260926 20:43:04.041106 6459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:43:04.043694 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 6459
I20260926 20:43:04.044627 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:38619
--local_ip_for_outbound_sockets=127.23.44.195
--tserver_master_addrs=127.23.44.254:45935
--webserver_port=33429
--webserver_interface=127.23.44.195
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:43:04.047853 6591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:04.045141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:44259)
0926 20:43:04.046271 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:04.046621 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":449,"threads_started":1}
W20260926 20:43:04.049113 6589 heartbeater.cc:644] Failed to heartbeat to 127.23.44.254:45935 (0 consecutive failures): Network error: Failed to ping master at 127.23.44.254:45935: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
I20260926 20:43:04.050333 6591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:04.049794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:44083)
0926 20:43:04.050012 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:04.050126 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260926 20:43:04.051244 6591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:04.050876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:55701)
0926 20:43:04.051034 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:04.051126 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 20:43:04.051471 6589 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 20:43:04.399740 6592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:04.400043 6592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:04.400141 6592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:04.407735 6592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:04.407927 6592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:04.408120 6592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:43:04.418217 6592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:38619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=33429
--tserver_master_addrs=127.23.44.254:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:04.419606 6592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:04.420990 6592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:04.435312 6600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:04.435611 6597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:04.435333 6598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:04.438024 6592 server_base.cc:1034] running on GCE node
I20260926 20:43:04.438786 6592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:04.439852 6592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:04.441387 6592 hybrid_clock.cc:648] HybridClock initialized: now 1790455384441305 us; error 69 us; skew 500 ppm
I20260926 20:43:04.441776 6592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:04 dist-test-slave-4d5t krb5kdc[5140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455384, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:04.446316 6592 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:43:04.448280 6592 webserver.cc:460] Webserver started at http://127.23.44.195:33429/ using document root <none> and password file <none>
I20260926 20:43:04.449173 6592 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:04.449334 6592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:04.455714 6592 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260926 20:43:04.459257 6607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:04.460927 6592 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:43:04.461136 6592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "387b20d842b6457b9ab32a65fd6c447c"
format_stamp: "Formatted at 2026-09-26 20:42:58 on dist-test-slave-4d5t"
server_key: "85d5a20e5a8218e834f4941a86d125fc"
server_key_iv: "3f31d2db53973ec96debc462d5d10936"
server_key_version: "encryptionkey@0"
I20260926 20:43:04.462214 6592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:04.484105 6592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:04.488399 6592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:04.488745 6592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:04.489782 6592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:43:04.491219 6592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:43:04.491346 6592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:04.491506 6592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:43:04.491603 6592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:04.530871 6592 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:38619
I20260926 20:43:04.530900 6719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:38619 every 8 connection(s)
I20260926 20:43:04.532512 6592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:43:04.533098 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 6592
I20260926 20:43:04.537943 6723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:04.536009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:53437)
0926 20:43:04.536838 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:04.537061 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":299,"threads_started":1}
W20260926 20:43:04.538767 6720 heartbeater.cc:644] Failed to heartbeat to 127.23.44.254:45935 (0 consecutive failures): Network error: Failed to ping master at 127.23.44.254:45935: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
I20260926 20:43:04.539831 6723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:04.539274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:48587)
0926 20:43:04.539540 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:04.539657 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260926 20:43:04.541059 6723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:04.540486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:38943)
0926 20:43:04.540784 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:04.540882 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
W20260926 20:43:04.541535 6720 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 20:43:04.581441 6724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:04.580395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:42749)
0926 20:43:04.580942 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:04.581069 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":278,"threads_started":1}
I20260926 20:43:04.823686 6725 raft_consensus.cc:482] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:43:04.824100 6725 raft_consensus.cc:504] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:04.825847 6725 leader_election.cc:288] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159), 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619)
Sep 26 20:43:04 dist-test-slave-4d5t krb5kdc[5140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455383, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
Sep 26 20:43:04 dist-test-slave-4d5t krb5kdc[5140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455383, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:43:04.855367 6727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:04.826397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:46185 (local address 127.23.44.194:36159)
0926 20:43:04.828770 (+ 2373us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:04.828800 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:04.828839 (+ 39us) server_negotiation.cc:407] Connection header received
0926 20:43:04.830402 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:04.830413 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:04.830826 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:04.831192 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:04.833408 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:04.834705 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:04.836659 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:04.837323 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:04.837551 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:04.847989 (+ 10438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:04.847998 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:04.848013 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:04.848056 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:04.851270 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:04.852272 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:04.852279 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:04.852285 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:04.852460 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:04.852986 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:04.852992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:04.852995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:04.853546 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:04.853743 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:04.854567 (+ 824us) server_negotiation.cc:299] Negotiation successful
0926 20:43:04.854896 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2214,"thread_start_us":2050,"threads_started":1}
I20260926 20:43:04.855429 6726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:04.826390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:38619 (local address 127.23.44.193:44921)
0926 20:43:04.828117 (+ 1727us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:04.828142 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:04.828270 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:04.831356 (+ 3086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:04.831370 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:04.832243 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:43:04.833159 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:04.833189 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:04.835017 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:04.835021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:04.836508 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:04.836525 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:04.837567 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:04.837577 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:04.837882 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:04.843980 (+ 6098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:43:04.844098 (+ 118us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:43:04.847621 (+ 3523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:43:04.851453 (+ 3832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:04.851463 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:04.851481 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:04.852100 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:04.852623 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:04.852628 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:04.852632 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:04.852868 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:04.853697 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:43:04.853705 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:43:04.854043 (+ 338us) client_negotiation.cc:769] Sending connection context
0926 20:43:04.854321 (+ 278us) client_negotiation.cc:240] Negotiation successful
0926 20:43:04.854936 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1558,"mutex_wait_us":161,"spinlock_wait_cycles":19968,"thread_start_us":1372,"threads_started":1}
I20260926 20:43:04.855422 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:04.826622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:44921 (local address 127.23.44.195:38619)
0926 20:43:04.828729 (+ 2107us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:04.828742 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:04.828814 (+ 72us) server_negotiation.cc:407] Connection header received
0926 20:43:04.830535 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:04.830544 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:04.830908 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:04.831263 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:04.833423 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:04.834802 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:04.836716 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:04.837336 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:04.837548 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:04.847986 (+ 10438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:04.847996 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:04.848013 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:04.848059 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:04.851253 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:04.852296 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:04.852303 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:04.852309 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:04.852475 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:04.853000 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:04.853011 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:04.853013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:04.853542 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:04.853736 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:04.854564 (+ 828us) server_negotiation.cc:299] Negotiation successful
0926 20:43:04.854968 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1861,"thread_start_us":1621,"threads_started":1}
I20260926 20:43:04.855865 6724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:04.826187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:36159 (local address 127.23.44.193:46185)
0926 20:43:04.826556 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:04.826594 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:04.827225 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:04.831356 (+ 4131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:04.831367 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:04.832201 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:43:04.833159 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:04.833189 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:04.834908 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:04.834917 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:04.836464 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:04.836480 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:04.837613 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:04.837621 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:04.837883 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:04.843858 (+ 5975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:43:04.843938 (+ 80us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:43:04.847621 (+ 3683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:43:04.851464 (+ 3843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:04.851480 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:04.851482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:04.852103 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:04.852647 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:04.852651 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:04.852654 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:04.852861 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:04.853697 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:43:04.853704 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:43:04.854043 (+ 339us) client_negotiation.cc:769] Sending connection context
0926 20:43:04.854321 (+ 278us) client_negotiation.cc:240] Negotiation successful
0926 20:43:04.855100 (+ 779us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":505,"spinlock_wait_cycles":44032}
W20260926 20:43:04.859942 6343 leader_election.cc:341] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159): Not found: Tablet not found: f81a5bf9e9f1405bb9fdf1c9745b6fe6
W20260926 20:43:04.860265 6342 leader_election.cc:341] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619): Not found: Tablet not found: f81a5bf9e9f1405bb9fdf1c9745b6fe6
I20260926 20:43:04.860487 6342 leader_election.cc:302] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aad6447897c84bbe8edb1874bb20c162; no voters: 387b20d842b6457b9ab32a65fd6c447c, 5bd22737c91d42c498f6643804dd1d71
I20260926 20:43:04.861209 6725 raft_consensus.cc:2726] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 20:43:05.053498 6729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:05.052265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:45939)
0926 20:43:05.053113 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:05.053256 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":431,"threads_started":1}
I20260926 20:43:05.543113 6730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:05.542181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:35261)
0926 20:43:05.542733 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:05.542881 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":290,"threads_started":1}
I20260926 20:43:05.583163 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:05.582455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:57637)
0926 20:43:05.582914 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:05.582995 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":256,"threads_started":1}
I20260926 20:43:06.055371 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:06.054598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:40271)
0926 20:43:06.055097 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:06.055190 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":271,"threads_started":1}
I20260926 20:43:06.545468 6734 raft_consensus.cc:482] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:43:06.545562 6733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:06.544255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:58769)
0926 20:43:06.545078 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:06.545326 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":311,"threads_started":1}
I20260926 20:43:06.545837 6734 raft_consensus.cc:504] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:06.546590 6734 leader_election.cc:288] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159), 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619)
W20260926 20:43:06.547479 6343 leader_election.cc:341] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159): Not found: Tablet not found: f81a5bf9e9f1405bb9fdf1c9745b6fe6
W20260926 20:43:06.547652 6342 leader_election.cc:341] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619): Not found: Tablet not found: f81a5bf9e9f1405bb9fdf1c9745b6fe6
I20260926 20:43:06.547724 6342 leader_election.cc:302] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aad6447897c84bbe8edb1874bb20c162; no voters: 387b20d842b6457b9ab32a65fd6c447c, 5bd22737c91d42c498f6643804dd1d71
I20260926 20:43:06.547936 6734 raft_consensus.cc:2726] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 20:43:06.585057 6735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:06.584141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:37821)
0926 20:43:06.584771 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:06.584868 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":308,"threads_started":1}
I20260926 20:43:07.058779 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:07.056881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:47667)
0926 20:43:07.058503 (+ 1622us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:07.058601 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1470,"thread_start_us":1318,"threads_started":1}
I20260926 20:43:07.547537 6737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:07.546484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:59929)
0926 20:43:07.547191 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:07.547337 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":343,"threads_started":1}
I20260926 20:43:07.586918 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:07.586147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:41039)
0926 20:43:07.586661 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:07.586752 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":295,"threads_started":1}
I20260926 20:43:08.061317 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:08.059970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:60831)
0926 20:43:08.060910 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:08.061074 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":332,"threads_started":1}
I20260926 20:43:08.549861 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:08.548868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:45807)
0926 20:43:08.549509 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:08.549618 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":310,"threads_started":1}
I20260926 20:43:08.589515 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:08.588690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:53105)
0926 20:43:08.589197 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:08.589304 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":270,"threads_started":1}
I20260926 20:43:08.608162 6742 raft_consensus.cc:482] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:43:08.608764 6742 raft_consensus.cc:504] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:08.609805 6742 leader_election.cc:288] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159), 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619)
W20260926 20:43:08.611224 6343 leader_election.cc:341] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159): Not found: Tablet not found: f81a5bf9e9f1405bb9fdf1c9745b6fe6
W20260926 20:43:08.611557 6342 leader_election.cc:341] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619): Not found: Tablet not found: f81a5bf9e9f1405bb9fdf1c9745b6fe6
I20260926 20:43:08.611683 6342 leader_election.cc:302] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aad6447897c84bbe8edb1874bb20c162; no voters: 387b20d842b6457b9ab32a65fd6c447c, 5bd22737c91d42c498f6643804dd1d71
I20260926 20:43:08.612146 6742 raft_consensus.cc:2726] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 20:43:09.063421 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:09.062620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:46569)
0926 20:43:09.063137 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:09.063228 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":293,"threads_started":1}
I20260926 20:43:09.551580 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:09.550829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:39919)
0926 20:43:09.551322 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:09.551414 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":279,"threads_started":1}
I20260926 20:43:09.591434 6745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:09.590568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:55609)
0926 20:43:09.591132 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:09.591238 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":329,"threads_started":1}
I20260926 20:43:10.065261 6746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:10.064431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:59759)
0926 20:43:10.065007 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:10.065083 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":351,"threads_started":1}
I20260926 20:43:10.553602 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:10.552737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:55121)
0926 20:43:10.553327 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:10.553418 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":290,"threads_started":1}
I20260926 20:43:10.593498 6748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:10.592469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:39321)
0926 20:43:10.593135 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:10.593261 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":352,"threads_started":1}
I20260926 20:43:11.067301 6749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:11.066291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:38175)
0926 20:43:11.067015 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:11.067106 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":377,"threads_started":1}
I20260926 20:43:11.556116 6750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:11.554684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:37965)
0926 20:43:11.555716 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:11.555845 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":531,"threads_started":1}
I20260926 20:43:11.595489 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:11.594603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:51921)
0926 20:43:11.595179 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:11.595302 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":320,"threads_started":1}
I20260926 20:43:12.069489 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:12.068716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:60593)
0926 20:43:12.069198 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:12.069285 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":239,"threads_started":1}
I20260926 20:43:12.558423 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:12.557545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:60413)
0926 20:43:12.558099 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:12.558225 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":288,"threads_started":1}
I20260926 20:43:12.597862 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:12.596910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:41741)
0926 20:43:12.597561 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:12.597674 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":358,"threads_started":1}
I20260926 20:43:13.071417 6755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:13.070568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:42305)
0926 20:43:13.071150 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:13.071242 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":293,"threads_started":1}
I20260926 20:43:13.178319 6756 raft_consensus.cc:482] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:43:13.178558 6756 raft_consensus.cc:504] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:13.179382 6756 leader_election.cc:288] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159), 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619)
W20260926 20:43:13.180434 6343 leader_election.cc:341] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159): Not found: Tablet not found: f81a5bf9e9f1405bb9fdf1c9745b6fe6
W20260926 20:43:13.180809 6342 leader_election.cc:341] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619): Not found: Tablet not found: f81a5bf9e9f1405bb9fdf1c9745b6fe6
I20260926 20:43:13.181051 6342 leader_election.cc:302] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aad6447897c84bbe8edb1874bb20c162; no voters: 387b20d842b6457b9ab32a65fd6c447c, 5bd22737c91d42c498f6643804dd1d71
I20260926 20:43:13.181530 6756 raft_consensus.cc:2726] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 20:43:13.560541 6757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:13.559549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:47207)
0926 20:43:13.560248 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:13.560346 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":323,"threads_started":1}
I20260926 20:43:13.600322 6758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:13.599283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:42897)
0926 20:43:13.599860 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:13.600021 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":297,"threads_started":1}
I20260926 20:43:14.073565 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:14.072555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:49659)
0926 20:43:14.073167 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:14.073286 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":305,"threads_started":1}
I20260926 20:43:14.562794 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:14.561921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:46165)
0926 20:43:14.562423 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:14.562521 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":262,"threads_started":1}
I20260926 20:43:14.602386 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:14.601590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:35075)
0926 20:43:14.602105 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:14.602195 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":290,"threads_started":1}
I20260926 20:43:15.075907 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:15.074897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:58169)
0926 20:43:15.075616 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:15.075733 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":420,"threads_started":1}
I20260926 20:43:15.565210 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:15.564116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:59675)
0926 20:43:15.564912 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:15.565030 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":365,"threads_started":1}
I20260926 20:43:15.604414 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:15.603586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:45569)
0926 20:43:15.604138 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:15.604225 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":325,"threads_started":1}
I20260926 20:43:16.078702 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:16.077250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:34517)
0926 20:43:16.078061 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:16.078455 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":389,"threads_started":1}
I20260926 20:43:16.567209 6766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:16.566411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:53839)
0926 20:43:16.566854 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:16.566940 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":238,"threads_started":1}
I20260926 20:43:16.606419 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:16.605376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:39975)
0926 20:43:16.606125 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:16.606233 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":322,"threads_started":1}
I20260926 20:43:17.081519 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:17.080178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:44843)
0926 20:43:17.080905 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:17.081272 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":392,"threads_started":1}
I20260926 20:43:17.569151 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:17.568270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:33959)
0926 20:43:17.568805 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:17.568936 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":307,"threads_started":1}
I20260926 20:43:17.608337 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:17.607548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:43227)
0926 20:43:17.608057 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:17.608152 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":284,"threads_started":1}
I20260926 20:43:18.083642 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:18.082613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:46725)
0926 20:43:18.083161 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:18.083338 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":294,"threads_started":1}
I20260926 20:43:18.570822 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:18.570030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:50165)
0926 20:43:18.570529 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:18.570621 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":265,"threads_started":1}
I20260926 20:43:18.610730 6773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:18.609869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:40053)
0926 20:43:18.610379 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:18.610471 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":263,"threads_started":1}
I20260926 20:43:19.085824 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:19.085030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.194:44901)
0926 20:43:19.085537 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:19.085655 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":274,"threads_started":1}
I20260926 20:43:19.467170 6775 raft_consensus.cc:482] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:43:19.467458 6775 raft_consensus.cc:504] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6447897c84bbe8edb1874bb20c162" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 38609 } } peers { permanent_uuid: "5bd22737c91d42c498f6643804dd1d71" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 36159 } } peers { permanent_uuid: "387b20d842b6457b9ab32a65fd6c447c" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 38619 } }
I20260926 20:43:19.468197 6775 leader_election.cc:288] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159), 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619)
W20260926 20:43:19.469551 6342 leader_election.cc:341] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 387b20d842b6457b9ab32a65fd6c447c (127.23.44.195:38619): Not found: Tablet not found: f81a5bf9e9f1405bb9fdf1c9745b6fe6
W20260926 20:43:19.469822 6343 leader_election.cc:341] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5bd22737c91d42c498f6643804dd1d71 (127.23.44.194:36159): Not found: Tablet not found: f81a5bf9e9f1405bb9fdf1c9745b6fe6
I20260926 20:43:19.470021 6343 leader_election.cc:302] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aad6447897c84bbe8edb1874bb20c162; no voters: 387b20d842b6457b9ab32a65fd6c447c, 5bd22737c91d42c498f6643804dd1d71
I20260926 20:43:19.470341 6775 raft_consensus.cc:2726] T f81a5bf9e9f1405bb9fdf1c9745b6fe6 P aad6447897c84bbe8edb1874bb20c162 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 20:43:19.543828 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 6292
I20260926 20:43:19.566793 6776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:19.545102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41370 (local address 127.23.44.193:38609)
0926 20:43:19.545879 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:19.545893 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:19.545928 (+ 35us) server_negotiation.cc:407] Connection header received
0926 20:43:19.545998 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:19.546005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:19.546355 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:19.546565 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:19.549522 (+ 2957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:19.550573 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:19.551819 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:19.552428 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:19.552604 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:19.560258 (+ 7654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:19.560273 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:19.560291 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:19.560327 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:19.563591 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:19.564430 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:19.564451 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:19.564453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:19.564656 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:19.565052 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:19.565058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:19.565062 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:19.565428 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:19.565771 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:19.566080 (+ 309us) server_negotiation.cc:299] Negotiation successful
0926 20:43:19.566394 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":344,"threads_started":1}
I20260926 20:43:19.568308 6451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:19.572965 6778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:19.572038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.195:48437)
0926 20:43:19.572607 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:19.572754 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":298,"threads_started":1}
I20260926 20:43:19.662091 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 6292
I20260926 20:43:19.662127 6779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:19.660940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45935 (local address 127.23.44.193:56783)
0926 20:43:19.661560 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:19.661810 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:45935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":268,"threads_started":1}
I20260926 20:43:19.682297 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 6459
I20260926 20:43:19.698120 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:19.683134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34918 (local address 127.23.44.194:36159)
0926 20:43:19.683620 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:19.683625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:19.683642 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:43:19.683698 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:19.683702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:19.683762 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:19.683857 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:19.685485 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:19.686736 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:19.688118 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:19.688926 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:19.689048 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:19.691790 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:19.691797 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:19.691800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:19.691836 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:19.694760 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:19.695548 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:19.695553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:19.695555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:19.695701 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:19.696071 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:19.696075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:19.696078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:19.696407 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:19.696743 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:19.697592 (+ 849us) server_negotiation.cc:299] Negotiation successful
0926 20:43:19.697810 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":282,"threads_started":1}
I20260926 20:43:19.698995 6584 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:19.787076 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 6459
I20260926 20:43:19.804943 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 6592
I20260926 20:43:19.818890 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:19.805812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42494 (local address 127.23.44.195:38619)
0926 20:43:19.806383 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:19.806387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:19.806403 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:43:19.806458 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:19.806462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:19.806528 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:19.806661 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:19.808090 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:19.808977 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:19.810163 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:19.810594 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:19.810737 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:19.813447 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:19.813452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:19.813455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:19.813489 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:19.816016 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:19.816834 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:19.816840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:19.816843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:19.816987 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:19.817379 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:19.817386 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:19.817389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:19.817662 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:19.817956 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:19.818284 (+ 328us) server_negotiation.cc:299] Negotiation successful
0926 20:43:19.818509 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":275,"threads_started":1}
I20260926 20:43:19.819708 6715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:19.904938 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 6592
2026-09-26T20:43:19Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25210 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:43:20 dist-test-slave-4d5t krb5kdc[6786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:43:20 dist-test-slave-4d5t krb5kdc[6786](info): set up 2 sockets
Sep 26 20:43:20 dist-test-slave-4d5t krb5kdc[6786](info): commencing operation
krb5kdc: starting...
W20260926 20:43:21.077584 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:43:21 dist-test-slave-4d5t krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:43:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:43:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:43:21.345976 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:44273
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:44273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:43:21.696471 6799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:21.696868 6799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:21.697075 6799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:21.705700 6799 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 20:43:21.705843 6799 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 20:43:21.705935 6799 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 20:43:21.706030 6799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:43:21.706112 6799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:21.706208 6799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:21.706310 6799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:43:21.706411 6799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:43:21.718459 6799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:44273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:44273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:21.719898 6799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:21.721436 6799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:21.736656 6805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:21.736729 6807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:21.736702 6804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:21.737463 6799 server_base.cc:1034] running on GCE node
I20260926 20:43:21.738404 6799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:21.741226 6799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:21.742484 6799 hybrid_clock.cc:648] HybridClock initialized: now 1790455401742393 us; error 64 us; skew 500 ppm
I20260926 20:43:21.742858 6799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:21 dist-test-slave-4d5t krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455401, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:21.746825 6799 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:43:21.748631 6799 webserver.cc:460] Webserver started at http://127.23.44.254:34431/ using document root <none> and password file <none>
I20260926 20:43:21.749475 6799 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:21.749612 6799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:21.749974 6799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:43:21.753631 6799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "0f9b50c7e5a44604a752c107c4dd51e9"
format_stamp: "Formatted at 2026-09-26 20:43:21 on dist-test-slave-4d5t"
server_key: "46681699cbdea15f6f2b04bafdcde52f"
server_key_iv: "954e30781421fadf4c9348e6110e7262"
server_key_version: "encryptionkey@0"
I20260926 20:43:21.754752 6799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "0f9b50c7e5a44604a752c107c4dd51e9"
format_stamp: "Formatted at 2026-09-26 20:43:21 on dist-test-slave-4d5t"
server_key: "46681699cbdea15f6f2b04bafdcde52f"
server_key_iv: "954e30781421fadf4c9348e6110e7262"
server_key_version: "encryptionkey@0"
I20260926 20:43:21.761708 6799 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260926 20:43:21.766964 6814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:21.768723 6799 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:43:21.768949 6799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "0f9b50c7e5a44604a752c107c4dd51e9"
format_stamp: "Formatted at 2026-09-26 20:43:21 on dist-test-slave-4d5t"
server_key: "46681699cbdea15f6f2b04bafdcde52f"
server_key_iv: "954e30781421fadf4c9348e6110e7262"
server_key_version: "encryptionkey@0"
I20260926 20:43:21.769223 6799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:21.787817 6799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:21.793056 6799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:21.793383 6799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:21.814311 6799 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:44273
I20260926 20:43:21.814334 6865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:44273 every 8 connection(s)
I20260926 20:43:21.815725 6799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:43:21.819784 6866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:43:21.824224 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 6799
I20260926 20:43:21.824492 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:43:21.825205 23731 external_mini_cluster.cc:1428] Setting key 6c423cb3e1f48b7545012e90d7e7cf05
I20260926 20:43:21.832522 6866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Bootstrap starting.
I20260926 20:43:21.836139 6866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Neither blocks nor log segments found. Creating new log.
I20260926 20:43:21.837531 6866 log.cc:824] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Log is configured to *not* fsync() on all Append() calls
I20260926 20:43:21.841235 6866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: No bootstrap required, opened a new log
Sep 26 20:43:21 dist-test-slave-4d5t krb5kdc[6786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:43:21.852211 6869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:21.827413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35292 (local address 127.23.44.254:44273)
0926 20:43:21.828315 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:21.828332 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:21.828377 (+ 45us) server_negotiation.cc:407] Connection header received
0926 20:43:21.830967 (+ 2590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:21.830997 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:21.831451 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:21.831896 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:21.833611 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:21.834960 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:21.836434 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:21.837250 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:21.837475 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:21.843446 (+ 5971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:21.843461 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:21.843478 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:21.843521 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:21.847150 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:21.848229 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:21.848237 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:21.848243 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:21.848439 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:21.848919 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:21.848923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:21.848926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:21.849575 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:21.849818 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:21.850467 (+ 649us) server_negotiation.cc:299] Negotiation successful
0926 20:43:21.850955 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":413,"threads_started":1}
I20260926 20:43:21.852135 6866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } }
I20260926 20:43:21.852690 6866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:43:21.852828 6866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f9b50c7e5a44604a752c107c4dd51e9, State: Initialized, Role: FOLLOWER
I20260926 20:43:21.853518 6866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } }
I20260926 20:43:21.853839 6866 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:43:21.854012 6866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:43:21.854239 6866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:43:21.857571 6866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } }
I20260926 20:43:21.858115 6866 leader_election.cc:302] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f9b50c7e5a44604a752c107c4dd51e9; no voters:
I20260926 20:43:21.858737 6866 leader_election.cc:288] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:43:21.858911 6871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:43:21.860347 6871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 1 LEADER]: Becoming Leader. State: Replica: 0f9b50c7e5a44604a752c107c4dd51e9, State: Running, Role: LEADER
I20260926 20:43:21.860983 6871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } }
I20260926 20:43:21.861389 6866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:43:21.864272 6872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } } }
I20260926 20:43:21.864729 6872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: This master's current role is: LEADER
I20260926 20:43:21.864485 6873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f9b50c7e5a44604a752c107c4dd51e9. Latest consensus state: current_term: 1 leader_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } } }
I20260926 20:43:21.864955 6873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: This master's current role is: LEADER
I20260926 20:43:21.870066 6877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:43:21.875777 6877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:43:21.885492 6877 catalog_manager.cc:1347] Generated new cluster ID: 6fba881e86c049d88167e8830155778d
I20260926 20:43:21.885632 6877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:43:21.902827 6877 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:43:21.904902 6877 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:43:21.916666 6877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Generated new TSK 0
I20260926 20:43:21.917829 6877 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:43:21.992910 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:44273
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:43:22.378259 6892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:22.378650 6892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:22.378822 6892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:22.387027 6892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:22.387219 6892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:22.387432 6892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:43:22.398380 6892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:44273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:22.399896 6892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:22.401636 6892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:22.416623 6898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:22.416623 6897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:22.416891 6900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:22.418213 6892 server_base.cc:1034] running on GCE node
I20260926 20:43:22.418942 6892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:22.420038 6892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:22.421298 6892 hybrid_clock.cc:648] HybridClock initialized: now 1790455402421252 us; error 30 us; skew 500 ppm
I20260926 20:43:22.421677 6892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:22 dist-test-slave-4d5t krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455402, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:22.425261 6892 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:43:22.426855 6892 webserver.cc:460] Webserver started at http://127.23.44.193:42985/ using document root <none> and password file <none>
I20260926 20:43:22.427630 6892 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:22.427800 6892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:22.428144 6892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:43:22.431572 6892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "6412b3674fcc4595bd37f96961c9dd3a"
format_stamp: "Formatted at 2026-09-26 20:43:22 on dist-test-slave-4d5t"
server_key: "73a8634c323270bd7b1bfea9473368ec"
server_key_iv: "bf17692a74433d3a8ba7b3158f5ed294"
server_key_version: "encryptionkey@0"
I20260926 20:43:22.432474 6892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "6412b3674fcc4595bd37f96961c9dd3a"
format_stamp: "Formatted at 2026-09-26 20:43:22 on dist-test-slave-4d5t"
server_key: "73a8634c323270bd7b1bfea9473368ec"
server_key_iv: "bf17692a74433d3a8ba7b3158f5ed294"
server_key_version: "encryptionkey@0"
I20260926 20:43:22.438450 6892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:43:22.442708 6907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:22.444165 6892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:43:22.444376 6892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "6412b3674fcc4595bd37f96961c9dd3a"
format_stamp: "Formatted at 2026-09-26 20:43:22 on dist-test-slave-4d5t"
server_key: "73a8634c323270bd7b1bfea9473368ec"
server_key_iv: "bf17692a74433d3a8ba7b3158f5ed294"
server_key_version: "encryptionkey@0"
I20260926 20:43:22.444669 6892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:22.468808 6892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:22.473981 6892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:22.474305 6892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:22.475378 6892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:43:22.476899 6892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:43:22.477011 6892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:22.477165 6892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:43:22.477239 6892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:22.515929 6892 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:46479
I20260926 20:43:22.515987 7019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:46479 every 8 connection(s)
I20260926 20:43:22.517273 6892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:43:22.523736 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 6892
I20260926 20:43:22.523999 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:43:22.524935 23731 external_mini_cluster.cc:1428] Setting key 5982496618185a975131d4836d1942c6
Sep 26 20:43:22 dist-test-slave-4d5t krb5kdc[6786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455402, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:43:22.545143 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:22.520036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:34565 (local address 127.23.44.254:44273)
0926 20:43:22.520536 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:22.520542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:22.521950 (+ 1408us) server_negotiation.cc:407] Connection header received
0926 20:43:22.525376 (+ 3426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:22.525382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:22.525471 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:22.525573 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:22.527836 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:22.528792 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:22.531796 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:22.532247 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:22.532370 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:22.537369 (+ 4999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:22.537378 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:22.537386 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:22.537437 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:22.541279 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:22.542187 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:22.542196 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:22.542203 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:22.542360 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:22.542880 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:22.542894 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:22.542900 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:22.543337 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:22.543499 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:22.544310 (+ 811us) server_negotiation.cc:299] Negotiation successful
0926 20:43:22.544635 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":227,"threads_started":1}
I20260926 20:43:22.546113 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:22.520610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:34565)
0926 20:43:22.521677 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:22.521743 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:22.524895 (+ 3152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:22.525866 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:22.525875 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:22.526671 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:43:22.527548 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:22.527577 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:22.529017 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:22.529027 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:22.531575 (+ 2548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:22.531598 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:22.532540 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:22.532578 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:22.532984 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:22.533897 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:43:22.533936 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:43:22.537045 (+ 3109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:43:22.541493 (+ 4448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:22.541502 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:22.541517 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:22.542041 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:22.542504 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:22.542508 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:22.542510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:22.542756 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:22.543499 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:43:22.543506 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:43:22.543896 (+ 390us) client_negotiation.cc:769] Sending connection context
0926 20:43:22.544167 (+ 271us) client_negotiation.cc:240] Negotiation successful
0926 20:43:22.544748 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":414,"threads_started":1}
I20260926 20:43:22.548020 7020 heartbeater.cc:347] Connected to a master server at 127.23.44.254:44273
I20260926 20:43:22.548379 7020 heartbeater.cc:464] Registering TS with master...
I20260926 20:43:22.549396 7020 heartbeater.cc:511] Master 127.23.44.254:44273 requested a full tablet report, sending...
I20260926 20:43:22.552068 6831 ts_manager.cc:194] Registered new tserver with Master: 6412b3674fcc4595bd37f96961c9dd3a (127.23.44.193:46479)
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
I20260926 20:43:22.554386 6831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:34565
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:43:22.588995 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:44273
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:43:22.940994 7026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:22.941309 7026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:22.941421 7026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:22.951285 7026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:22.951435 7026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:22.951566 7026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:43:22.962869 7026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:44273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:22.964324 7026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:22.965831 7026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:22.981191 7031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:22.981258 7034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:22.981221 7032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:22.982084 7026 server_base.cc:1034] running on GCE node
I20260926 20:43:22.982722 7026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:22.983822 7026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:22.985078 7026 hybrid_clock.cc:648] HybridClock initialized: now 1790455402984992 us; error 74 us; skew 500 ppm
I20260926 20:43:22.985466 7026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:22 dist-test-slave-4d5t krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455402, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:22.989054 7026 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:43:22.990718 7026 webserver.cc:460] Webserver started at http://127.23.44.194:42103/ using document root <none> and password file <none>
I20260926 20:43:22.991428 7026 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:22.991575 7026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:22.991882 7026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:43:22.994903 7026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "225cc05acc474d20927e9b25885d47f1"
format_stamp: "Formatted at 2026-09-26 20:43:22 on dist-test-slave-4d5t"
server_key: "71adafdb2034e3995aff77dfaca994f7"
server_key_iv: "d66c35bae1adac2cff1a7bea8f73d73e"
server_key_version: "encryptionkey@0"
I20260926 20:43:22.995659 7026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "225cc05acc474d20927e9b25885d47f1"
format_stamp: "Formatted at 2026-09-26 20:43:22 on dist-test-slave-4d5t"
server_key: "71adafdb2034e3995aff77dfaca994f7"
server_key_iv: "d66c35bae1adac2cff1a7bea8f73d73e"
server_key_version: "encryptionkey@0"
I20260926 20:43:23.001727 7026 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 20:43:23.006135 7041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:23.007748 7026 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:43:23.007956 7026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "225cc05acc474d20927e9b25885d47f1"
format_stamp: "Formatted at 2026-09-26 20:43:22 on dist-test-slave-4d5t"
server_key: "71adafdb2034e3995aff77dfaca994f7"
server_key_iv: "d66c35bae1adac2cff1a7bea8f73d73e"
server_key_version: "encryptionkey@0"
I20260926 20:43:23.008176 7026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:23.034832 7026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:23.039696 7026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:23.040040 7026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:23.041165 7026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:43:23.042573 7026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:43:23.042683 7026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:23.042836 7026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:43:23.042908 7026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:23.084717 7026 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:35231
I20260926 20:43:23.084736 7153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:35231 every 8 connection(s)
I20260926 20:43:23.086364 7026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:43:23.096067 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 7026
I20260926 20:43:23.096269 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:43:23.096901 23731 external_mini_cluster.cc:1428] Setting key 5b8785f10a1ec9b370d55df58683bedd
Sep 26 20:43:23 dist-test-slave-4d5t krb5kdc[6786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455402, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:43:23.109005 7156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:23.089345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:33471 (local address 127.23.44.254:44273)
0926 20:43:23.089893 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:23.089898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:23.090862 (+ 964us) server_negotiation.cc:407] Connection header received
0926 20:43:23.092754 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:23.092762 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:23.092859 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:23.092989 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:23.094521 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.095270 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.097648 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.098267 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.098406 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:23.102809 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:23.102816 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:23.102819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:23.102859 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:23.105740 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:23.106545 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:23.106552 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:23.106560 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:23.106686 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:23.107143 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:23.107148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:23.107150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:23.107435 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:23.107587 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:23.108334 (+ 747us) server_negotiation.cc:299] Negotiation successful
0926 20:43:23.108621 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":283,"threads_started":1}
I20260926 20:43:23.109970 7157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:23.089903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:33471)
0926 20:43:23.090685 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:23.090729 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:23.092452 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:23.093227 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:23.093235 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:23.093760 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:43:23.094332 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:23.094349 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.095443 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.095447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:23.097483 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:23.097504 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.098503 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.098513 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:23.098755 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:23.099487 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:43:23.099520 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:43:23.102500 (+ 2980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:43:23.105907 (+ 3407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:23.105916 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:23.105935 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:23.106403 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:23.106809 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:23.106814 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:23.106818 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:23.107010 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:23.107575 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:43:23.107583 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:43:23.107957 (+ 374us) client_negotiation.cc:769] Sending connection context
0926 20:43:23.108231 (+ 274us) client_negotiation.cc:240] Negotiation successful
0926 20:43:23.108827 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":313,"threads_started":1}
I20260926 20:43:23.111537 7154 heartbeater.cc:347] Connected to a master server at 127.23.44.254:44273
I20260926 20:43:23.111893 7154 heartbeater.cc:464] Registering TS with master...
I20260926 20:43:23.112876 7154 heartbeater.cc:511] Master 127.23.44.254:44273 requested a full tablet report, sending...
I20260926 20:43:23.114674 6831 ts_manager.cc:194] Registered new tserver with Master: 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231)
I20260926 20:43:23.116036 6831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:33471
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:43:23.161515 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:44273
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:43:23.517066 7160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:23.517341 7160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:23.517516 7160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:23.525442 7160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:23.525605 7160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:23.525802 7160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:43:23.537700 7160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:44273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:23.539193 7160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:23.540729 7160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:23.554991 7165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:23.554956 7168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:23.554924 7166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:23.556608 7160 server_base.cc:1034] running on GCE node
I20260926 20:43:23.557345 7160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:23.558495 7160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:23.558657 7020 heartbeater.cc:503] Master 127.23.44.254:44273 was elected leader, sending a full tablet report...
I20260926 20:43:23.559728 7160 hybrid_clock.cc:648] HybridClock initialized: now 1790455403559683 us; error 57 us; skew 500 ppm
I20260926 20:43:23.560083 7160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:23 dist-test-slave-4d5t krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455403, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:23.563817 7160 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:43:23.565697 7160 webserver.cc:460] Webserver started at http://127.23.44.195:42027/ using document root <none> and password file <none>
I20260926 20:43:23.566399 7160 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:23.566519 7160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:23.566859 7160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:43:23.570663 7160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "2e01cf8bdc2f4717b7a156574501b253"
format_stamp: "Formatted at 2026-09-26 20:43:23 on dist-test-slave-4d5t"
server_key: "339550196f560b5757b900dc042bd51c"
server_key_iv: "25000752c24ecf76486c33960b210fdc"
server_key_version: "encryptionkey@0"
I20260926 20:43:23.571517 7160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "2e01cf8bdc2f4717b7a156574501b253"
format_stamp: "Formatted at 2026-09-26 20:43:23 on dist-test-slave-4d5t"
server_key: "339550196f560b5757b900dc042bd51c"
server_key_iv: "25000752c24ecf76486c33960b210fdc"
server_key_version: "encryptionkey@0"
I20260926 20:43:23.578082 7160 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260926 20:43:23.581952 7175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:23.583463 7160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260926 20:43:23.583678 7160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "2e01cf8bdc2f4717b7a156574501b253"
format_stamp: "Formatted at 2026-09-26 20:43:23 on dist-test-slave-4d5t"
server_key: "339550196f560b5757b900dc042bd51c"
server_key_iv: "25000752c24ecf76486c33960b210fdc"
server_key_version: "encryptionkey@0"
I20260926 20:43:23.583932 7160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:23.612344 7160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:23.617278 7160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:23.617615 7160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:23.618752 7160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:43:23.620376 7160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:43:23.620503 7160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:23.620692 7160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:43:23.620774 7160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:23.660652 7160 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:41499
I20260926 20:43:23.660755 7287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:41499 every 8 connection(s)
I20260926 20:43:23.662019 7160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:43:23.671254 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 7160
I20260926 20:43:23.671479 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:43:23.672077 23731 external_mini_cluster.cc:1428] Setting key 19bf7a33457c217d7d932af62e01ff36
Sep 26 20:43:23 dist-test-slave-4d5t krb5kdc[6786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455403, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:43:23.686545 7290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:23.665635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:51005 (local address 127.23.44.254:44273)
0926 20:43:23.666155 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:23.666163 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:23.667299 (+ 1136us) server_negotiation.cc:407] Connection header received
0926 20:43:23.669311 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:23.669317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:23.669411 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:23.669518 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:23.672222 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.673122 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.675310 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.675727 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.675826 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:23.679656 (+ 3830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:23.679662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:23.679666 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:23.679710 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:23.682779 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:23.683778 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:23.683783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:23.683786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:23.683940 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:23.684443 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:23.684448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:23.684451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:23.684864 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:23.685008 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:23.685872 (+ 864us) server_negotiation.cc:299] Negotiation successful
0926 20:43:23.686139 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":249,"threads_started":1}
I20260926 20:43:23.687621 7291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:23.666269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:51005)
0926 20:43:23.667130 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:23.667186 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:23.669014 (+ 1828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:23.669960 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:23.669973 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:23.670994 (+ 1021us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:43:23.671972 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:23.672003 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.673313 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.673317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:23.675157 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:23.675178 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.675940 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.675946 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:23.676156 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:23.676921 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:43:23.676946 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:43:23.679376 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:43:23.682953 (+ 3577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:23.682962 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:23.682979 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:23.683625 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:23.684089 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:23.684093 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:23.684096 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:23.684289 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:23.685030 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:43:23.685037 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:43:23.685409 (+ 372us) client_negotiation.cc:769] Sending connection context
0926 20:43:23.685762 (+ 353us) client_negotiation.cc:240] Negotiation successful
0926 20:43:23.686320 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":332,"threads_started":1}
I20260926 20:43:23.689165 7288 heartbeater.cc:347] Connected to a master server at 127.23.44.254:44273
I20260926 20:43:23.689499 7288 heartbeater.cc:464] Registering TS with master...
I20260926 20:43:23.690196 7288 heartbeater.cc:511] Master 127.23.44.254:44273 requested a full tablet report, sending...
I20260926 20:43:23.691892 6831 ts_manager.cc:194] Registered new tserver with Master: 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499)
I20260926 20:43:23.692883 6831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:51005
I20260926 20:43:23.701357 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:43:23.720692 7290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:23.706289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35300 (local address 127.23.44.254:44273)
0926 20:43:23.706468 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:23.706473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:23.706842 (+ 369us) server_negotiation.cc:407] Connection header received
0926 20:43:23.706970 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:23.706974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:23.707060 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:23.707151 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:23.709048 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.709883 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.711574 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.712105 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.712293 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:23.714967 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:23.714985 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:23.714989 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:23.715028 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:23.717960 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:23.718670 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:23.718674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:23.718676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:23.718815 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:23.719279 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:23.719282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:23.719284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:23.719663 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:23.719854 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:23.720201 (+ 347us) server_negotiation.cc:299] Negotiation successful
0926 20:43:23.720379 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:43:23.725194 6831 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:43:23.732815 6831 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:43:23.774767 7302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:23.747493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:41499 (local address 127.0.0.1:47638)
0926 20:43:23.760879 (+ 13386us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:23.760957 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:23.761261 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:23.763223 (+ 1962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:23.763235 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:23.763294 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:23.763845 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:23.763862 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.766919 (+ 3057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.766928 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:23.770509 (+ 3581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:23.770536 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.773421 (+ 2885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.773432 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:23.773791 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:23.773880 (+ 89us) client_negotiation.cc:769] Sending connection context
0926 20:43:23.774007 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 20:43:23.774231 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12954,"thread_start_us":14406,"threads_started":1}
I20260926 20:43:23.775162 7303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:23.747878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47638 (local address 127.23.44.195:41499)
0926 20:43:23.762388 (+ 14510us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:23.762397 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:23.762432 (+ 35us) server_negotiation.cc:407] Connection header received
0926 20:43:23.762534 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:23.762544 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:23.762913 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:23.764085 (+ 1172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:23.764451 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.766695 (+ 2244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.770886 (+ 4191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.773118 (+ 2232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.774121 (+ 1003us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:23.774486 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:23.774581 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 20:43:23.774773 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14319,"thread_start_us":14120,"threads_started":1}
I20260926 20:43:23.778918 7304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:23.748063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59426 (local address 127.23.44.194:35231)
0926 20:43:23.759448 (+ 11385us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:23.759465 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:23.763074 (+ 3609us) server_negotiation.cc:407] Connection header received
0926 20:43:23.763470 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:23.763483 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:23.763796 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:23.764009 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:23.767206 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.770216 (+ 3010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.776144 (+ 5928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.777705 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.777852 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:23.778052 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:23.778443 (+ 391us) server_negotiation.cc:299] Negotiation successful
0926 20:43:23.778612 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11093,"thread_start_us":11427,"threads_started":1}
I20260926 20:43:23.779104 7301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:23.747654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:35231 (local address 127.0.0.1:59426)
0926 20:43:23.762922 (+ 15268us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:23.762948 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:23.763229 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:23.764411 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:23.764419 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:23.764459 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:23.766858 (+ 2399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:23.766879 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.771698 (+ 4819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.771708 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:23.775961 (+ 4253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:23.775979 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.777946 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.777955 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:23.778182 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:23.778238 (+ 56us) client_negotiation.cc:769] Sending connection context
0926 20:43:23.778342 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 20:43:23.778511 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15132,"thread_start_us":12573,"threads_started":1}
I20260926 20:43:23.779569 7223 tablet_service.cc:1467] Processing CreateTablet for tablet a1dea3d36f03438d86371d047d16d70e (DEFAULT_TABLE table=test-table [id=a7a6b050bb9e4457b949b1c0f96056c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:43:23.781177 7223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1dea3d36f03438d86371d047d16d70e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:43:23.783106 7089 tablet_service.cc:1467] Processing CreateTablet for tablet a1dea3d36f03438d86371d047d16d70e (DEFAULT_TABLE table=test-table [id=a7a6b050bb9e4457b949b1c0f96056c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:43:23.785022 7306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:23.762765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57698 (local address 127.23.44.193:46479)
0926 20:43:23.773450 (+ 10685us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:23.773460 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:23.774520 (+ 1060us) server_negotiation.cc:407] Connection header received
0926 20:43:23.775975 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:23.775988 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:23.776279 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:23.776540 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:23.777155 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.778644 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.782183 (+ 3539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.783621 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.783774 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:23.784000 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:23.784500 (+ 500us) server_negotiation.cc:299] Negotiation successful
0926 20:43:23.784689 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10426,"thread_start_us":10675,"threads_started":1}
I20260926 20:43:23.785293 7089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1dea3d36f03438d86371d047d16d70e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:43:23.785617 7305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:23.762598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:46479 (local address 127.0.0.1:57698)
0926 20:43:23.774347 (+ 11749us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:23.774377 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:23.774663 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:23.776500 (+ 1837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:23.776506 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:23.776542 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:23.776972 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:23.776984 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.778864 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.778878 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:23.781979 (+ 3101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:23.782000 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:23.783937 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:23.783946 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:23.784161 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:23.784215 (+ 54us) client_negotiation.cc:769] Sending connection context
0926 20:43:23.784439 (+ 224us) client_negotiation.cc:240] Negotiation successful
0926 20:43:23.784905 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11402,"thread_start_us":11221,"threads_started":1}
I20260926 20:43:23.790055 6955 tablet_service.cc:1467] Processing CreateTablet for tablet a1dea3d36f03438d86371d047d16d70e (DEFAULT_TABLE table=test-table [id=a7a6b050bb9e4457b949b1c0f96056c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:43:23.791610 6955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1dea3d36f03438d86371d047d16d70e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:43:23.794162 7307 tablet_bootstrap.cc:492] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253: Bootstrap starting.
I20260926 20:43:23.797791 7307 tablet_bootstrap.cc:654] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253: Neither blocks nor log segments found. Creating new log.
I20260926 20:43:23.798867 7307 log.cc:824] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253: Log is configured to *not* fsync() on all Append() calls
I20260926 20:43:23.801716 7309 tablet_bootstrap.cc:492] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1: Bootstrap starting.
I20260926 20:43:23.802277 7307 tablet_bootstrap.cc:492] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253: No bootstrap required, opened a new log
I20260926 20:43:23.802567 7307 ts_tablet_manager.cc:1397] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 20:43:23.805645 7309 tablet_bootstrap.cc:654] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1: Neither blocks nor log segments found. Creating new log.
I20260926 20:43:23.806309 7310 tablet_bootstrap.cc:492] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a: Bootstrap starting.
I20260926 20:43:23.807060 7309 log.cc:824] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1: Log is configured to *not* fsync() on all Append() calls
I20260926 20:43:23.809700 7310 tablet_bootstrap.cc:654] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a: Neither blocks nor log segments found. Creating new log.
I20260926 20:43:23.810518 7309 tablet_bootstrap.cc:492] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1: No bootstrap required, opened a new log
I20260926 20:43:23.810802 7309 ts_tablet_manager.cc:1397] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260926 20:43:23.811379 7310 log.cc:824] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a: Log is configured to *not* fsync() on all Append() calls
I20260926 20:43:23.813311 7307 raft_consensus.cc:348] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:23.813796 7307 raft_consensus.cc:374] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:43:23.813902 7307 raft_consensus.cc:729] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e01cf8bdc2f4717b7a156574501b253, State: Initialized, Role: FOLLOWER
I20260926 20:43:23.814664 7307 consensus_queue.cc:260] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:23.815048 7310 tablet_bootstrap.cc:492] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a: No bootstrap required, opened a new log
I20260926 20:43:23.815346 7310 ts_tablet_manager.cc:1397] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260926 20:43:23.821933 7288 heartbeater.cc:503] Master 127.23.44.254:44273 was elected leader, sending a full tablet report...
I20260926 20:43:23.823067 7307 ts_tablet_manager.cc:1428] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253: Time spent starting tablet: real 0.020s user 0.009s sys 0.011s
I20260926 20:43:23.823630 7309 raft_consensus.cc:348] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:23.824122 7309 raft_consensus.cc:374] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:43:23.824223 7309 raft_consensus.cc:729] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 225cc05acc474d20927e9b25885d47f1, State: Initialized, Role: FOLLOWER
I20260926 20:43:23.824872 7309 consensus_queue.cc:260] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:23.827134 7310 raft_consensus.cc:348] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:23.827641 7310 raft_consensus.cc:374] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:43:23.827800 7310 raft_consensus.cc:729] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6412b3674fcc4595bd37f96961c9dd3a, State: Initialized, Role: FOLLOWER
I20260926 20:43:23.828534 7310 consensus_queue.cc:260] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:23.832177 7154 heartbeater.cc:503] Master 127.23.44.254:44273 was elected leader, sending a full tablet report...
I20260926 20:43:23.833163 7309 ts_tablet_manager.cc:1428] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1: Time spent starting tablet: real 0.022s user 0.015s sys 0.008s
I20260926 20:43:23.834875 7310 ts_tablet_manager.cc:1428] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a: Time spent starting tablet: real 0.019s user 0.007s sys 0.010s
W20260926 20:43:23.838599 7155 tablet.cc:2369] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:43:23.913990 7289 tablet.cc:2369] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:43:24.020141 7021 tablet.cc:2369] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:43:24.112962 7313 raft_consensus.cc:482] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:43:24.113337 7313 raft_consensus.cc:504] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:24.115093 7313 leader_election.cc:288] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231), 6412b3674fcc4595bd37f96961c9dd3a (127.23.44.193:46479)
I20260926 20:43:24.122336 7306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:24.115604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:58715 (local address 127.23.44.193:46479)
0926 20:43:24.115823 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:24.115828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:24.116074 (+ 246us) server_negotiation.cc:407] Connection header received
0926 20:43:24.116320 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:24.116324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:24.116384 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:24.116486 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:24.116973 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.118193 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.120046 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.121394 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.121537 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:24.121596 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:24.121956 (+ 360us) server_negotiation.cc:299] Negotiation successful
0926 20:43:24.122061 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:43:24.122383 7291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:24.115586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:46479 (local address 127.23.44.195:58715)
0926 20:43:24.115968 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:24.115984 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:24.116151 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:24.116507 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:24.116512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:24.116540 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:24.116776 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:24.116798 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.118337 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.118342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:24.119867 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:24.119880 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.121566 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.121572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:24.121771 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:24.121819 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 20:43:24.121924 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 20:43:24.122075 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266}
I20260926 20:43:24.123170 7304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:24.115604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:39719 (local address 127.23.44.194:35231)
0926 20:43:24.115818 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:24.115824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:24.116680 (+ 856us) server_negotiation.cc:407] Connection header received
0926 20:43:24.116903 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:24.116908 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:24.116991 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:24.117057 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:24.117860 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.118768 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.121079 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.122145 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.122304 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:24.122380 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:24.122843 (+ 463us) server_negotiation.cc:299] Negotiation successful
0926 20:43:24.122987 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 20:43:24.123274 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:24.115761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:35231 (local address 127.23.44.195:39719)
0926 20:43:24.116519 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:24.116537 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:24.116754 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:24.117121 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:24.117126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:24.117158 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:24.117627 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:24.117642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.118960 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.118965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:24.120890 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:24.120907 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.122388 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.122396 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:24.122598 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:24.122631 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 20:43:24.122772 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 20:43:24.122890 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":447,"threads_started":1}
I20260926 20:43:24.123647 6975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1dea3d36f03438d86371d047d16d70e" candidate_uuid: "2e01cf8bdc2f4717b7a156574501b253" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6412b3674fcc4595bd37f96961c9dd3a" is_pre_election: true
I20260926 20:43:24.124214 6975 raft_consensus.cc:2454] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e01cf8bdc2f4717b7a156574501b253 in term 0.
I20260926 20:43:24.124223 7109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1dea3d36f03438d86371d047d16d70e" candidate_uuid: "2e01cf8bdc2f4717b7a156574501b253" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "225cc05acc474d20927e9b25885d47f1" is_pre_election: true
I20260926 20:43:24.124713 7109 raft_consensus.cc:2454] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e01cf8bdc2f4717b7a156574501b253 in term 0.
I20260926 20:43:24.125072 7178 leader_election.cc:302] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e01cf8bdc2f4717b7a156574501b253, 6412b3674fcc4595bd37f96961c9dd3a; no voters:
I20260926 20:43:24.125485 7313 raft_consensus.cc:2781] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:43:24.125638 7313 raft_consensus.cc:482] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:43:24.125744 7313 raft_consensus.cc:3037] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:43:24.127928 7313 raft_consensus.cc:504] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:24.128531 7313 leader_election.cc:288] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [CANDIDATE]: Term 1 election: Requested vote from peers 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231), 6412b3674fcc4595bd37f96961c9dd3a (127.23.44.193:46479)
I20260926 20:43:24.129062 7109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1dea3d36f03438d86371d047d16d70e" candidate_uuid: "2e01cf8bdc2f4717b7a156574501b253" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "225cc05acc474d20927e9b25885d47f1"
I20260926 20:43:24.129128 6975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1dea3d36f03438d86371d047d16d70e" candidate_uuid: "2e01cf8bdc2f4717b7a156574501b253" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6412b3674fcc4595bd37f96961c9dd3a"
I20260926 20:43:24.129262 7109 raft_consensus.cc:3037] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:43:24.129417 6975 raft_consensus.cc:3037] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:43:24.131567 7109 raft_consensus.cc:2454] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e01cf8bdc2f4717b7a156574501b253 in term 1.
I20260926 20:43:24.131682 6975 raft_consensus.cc:2454] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e01cf8bdc2f4717b7a156574501b253 in term 1.
I20260926 20:43:24.132051 7179 leader_election.cc:302] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 225cc05acc474d20927e9b25885d47f1, 2e01cf8bdc2f4717b7a156574501b253; no voters:
I20260926 20:43:24.132401 7313 raft_consensus.cc:2781] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:43:24.132949 7313 raft_consensus.cc:686] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [term 1 LEADER]: Becoming Leader. State: Replica: 2e01cf8bdc2f4717b7a156574501b253, State: Running, Role: LEADER
I20260926 20:43:24.133446 7313 consensus_queue.cc:237] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:24.138042 6829 catalog_manager.cc:5495] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195). New cstate: current_term: 1 leader_uuid: "2e01cf8bdc2f4717b7a156574501b253" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } health_report { overall_health: UNKNOWN } } }
I20260926 20:43:24.244812 7303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:24.238063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47650 (local address 127.23.44.195:41499)
0926 20:43:24.238303 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:24.238308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:24.238853 (+ 545us) server_negotiation.cc:407] Connection header received
0926 20:43:24.239033 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:24.239038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:24.239122 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:24.239272 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:43:24.239770 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.240498 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.242477 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.242911 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.243014 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:24.243610 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:43:24.244136 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:43:24.244261 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:24.244443 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 20:43:24.244525 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 20:43:24.253780 6975 raft_consensus.cc:1261] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Refusing update from remote peer 2e01cf8bdc2f4717b7a156574501b253: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:43:24.253789 7109 raft_consensus.cc:1261] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1 [term 1 FOLLOWER]: Refusing update from remote peer 2e01cf8bdc2f4717b7a156574501b253: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:43:24.254782 7317 consensus_queue.cc:1035] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:43:24.255214 7313 consensus_queue.cc:1035] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [LEADER]: Connected to new peer: Peer: permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:43:24.287710 6829 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:43:24.288031 6829 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:43:24.292155 6829 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Sending DeleteTablet for 3 replicas of tablet a1dea3d36f03438d86371d047d16d70e
I20260926 20:43:24.293416 7223 tablet_service.cc:1514] Processing DeleteTablet for tablet a1dea3d36f03438d86371d047d16d70e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:43:24 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:47638
I20260926 20:43:24.293442 6955 tablet_service.cc:1514] Processing DeleteTablet for tablet a1dea3d36f03438d86371d047d16d70e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:43:24 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:57698
I20260926 20:43:24.293884 7089 tablet_service.cc:1514] Processing DeleteTablet for tablet a1dea3d36f03438d86371d047d16d70e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:43:24 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:59426
I20260926 20:43:24.296896 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 6892
I20260926 20:43:24.306046 7331 tablet_replica.cc:330] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253: stopping tablet replica
I20260926 20:43:24.306645 7331 raft_consensus.cc:2229] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:43:24.307552 7331 raft_consensus.cc:2258] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:43:24.312355 7331 ts_tablet_manager.cc:1905] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:43:24.320880 7332 tablet_replica.cc:330] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a: stopping tablet replica
I20260926 20:43:24.321264 7333 tablet_replica.cc:330] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1: stopping tablet replica
I20260926 20:43:24.321408 7332 raft_consensus.cc:2229] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:43:24.321637 7331 ts_tablet_manager.cc:1918] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:43:24.321828 7331 log.cc:1196] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/a1dea3d36f03438d86371d047d16d70e
I20260926 20:43:24.322005 7332 raft_consensus.cc:2258] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:43:24.321859 7333 raft_consensus.cc:2229] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:43:24.322589 7333 raft_consensus.cc:2258] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:43:24.322759 7331 ts_tablet_manager.cc:1939] T a1dea3d36f03438d86371d047d16d70e P 2e01cf8bdc2f4717b7a156574501b253: Deleting consensus metadata
I20260926 20:43:24.324882 7333 ts_tablet_manager.cc:1905] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:43:24.324851 6817 catalog_manager.cc:4841] TS 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499): tablet a1dea3d36f03438d86371d047d16d70e (table test-table [id=a7a6b050bb9e4457b949b1c0f96056c4]) successfully deleted
I20260926 20:43:24.326994 7332 ts_tablet_manager.cc:1905] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:43:24.332208 7333 ts_tablet_manager.cc:1918] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:43:24.332468 7333 log.cc:1196] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/a1dea3d36f03438d86371d047d16d70e
I20260926 20:43:24.332568 7332 ts_tablet_manager.cc:1918] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:43:24.332756 7332 log.cc:1196] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/a1dea3d36f03438d86371d047d16d70e
I20260926 20:43:24.333189 7333 ts_tablet_manager.cc:1939] T a1dea3d36f03438d86371d047d16d70e P 225cc05acc474d20927e9b25885d47f1: Deleting consensus metadata
I20260926 20:43:24.333299 7332 ts_tablet_manager.cc:1939] T a1dea3d36f03438d86371d047d16d70e P 6412b3674fcc4595bd37f96961c9dd3a: Deleting consensus metadata
Sep 26 20:43:24 dist-test-slave-4d5t krb5kdc[6786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:43:24.335412 6818 catalog_manager.cc:4841] TS 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231): tablet a1dea3d36f03438d86371d047d16d70e (table test-table [id=a7a6b050bb9e4457b949b1c0f96056c4]) successfully deleted
I20260926 20:43:24.336048 6817 catalog_manager.cc:4841] TS 6412b3674fcc4595bd37f96961c9dd3a (127.23.44.193:46479): tablet a1dea3d36f03438d86371d047d16d70e (table test-table [id=a7a6b050bb9e4457b949b1c0f96056c4]) successfully deleted
I20260926 20:43:24.343240 7306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:24.298277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57702 (local address 127.23.44.193:46479)
0926 20:43:24.298507 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:24.298516 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:24.317711 (+ 19195us) server_negotiation.cc:407] Connection header received
0926 20:43:24.318243 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:24.318263 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:24.318353 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:24.318436 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:24.322429 (+ 3993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.323586 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.326727 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.327404 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.327514 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:24.337054 (+ 9540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:24.337066 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:24.337079 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:24.337114 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:24.339886 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:24.340902 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:24.340910 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:24.340913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:24.341091 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:24.341537 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:24.341540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:24.341543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:24.341954 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:24.342234 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:24.342644 (+ 410us) server_negotiation.cc:299] Negotiation successful
0926 20:43:24.342839 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:43:24.344022 7015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:24.463410 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 6892
I20260926 20:43:24.481797 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 7026
Sep 26 20:43:24 dist-test-slave-4d5t krb5kdc[6786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:43:24.500610 7304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:24.482737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59432 (local address 127.23.44.194:35231)
0926 20:43:24.482915 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:24.482921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:24.482941 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:43:24.483125 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:24.483129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:24.483208 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:24.483302 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:24.485202 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.486059 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.488216 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.488923 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.489042 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:24.494157 (+ 5115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:24.494174 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:24.494191 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:24.494235 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:24.497369 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:24.498290 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:24.498297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:24.498299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:24.498438 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:24.498894 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:24.498898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:24.498900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:24.499346 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:24.499585 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:24.500044 (+ 459us) server_negotiation.cc:299] Negotiation successful
0926 20:43:24.500215 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 20:43:24.501384 7149 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:24.625334 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 7026
I20260926 20:43:24.643106 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 7160
Sep 26 20:43:24 dist-test-slave-4d5t krb5kdc[6786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:43:24.669343 7303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:24.644063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47666 (local address 127.23.44.195:41499)
0926 20:43:24.644243 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:24.644248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:24.644439 (+ 191us) server_negotiation.cc:407] Connection header received
0926 20:43:24.644623 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:24.644628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:24.644693 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:24.644781 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:24.646934 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.648141 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.650438 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:24.650980 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:24.651103 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:24.661033 (+ 9930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:24.661049 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:24.661066 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:24.661121 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:24.665535 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:24.666621 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:24.666630 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:24.666633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:24.666866 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:24.667394 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:24.667402 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:24.667412 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:24.667939 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:24.668299 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:24.668663 (+ 364us) server_negotiation.cc:299] Negotiation successful
0926 20:43:24.668922 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:43:24.670101 7283 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:24.801904 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 7160
I20260926 20:43:24.823197 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 6799
I20260926 20:43:24.824466 6861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:24.940026 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 6799
I20260926 20:43:44.962313 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:44273
--webserver_interface=127.23.44.254
--webserver_port=34431
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:44273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:43:45.317075 7339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:45.317392 7339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:45.317560 7339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:45.325163 7339 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 20:43:45.325291 7339 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 20:43:45.325332 7339 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 20:43:45.325457 7339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:43:45.325560 7339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:45.325681 7339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:45.325776 7339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:43:45.325860 7339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:43:45.336320 7339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:44273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:44273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=34431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:45.337837 7339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:45.339252 7339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:45.365087 7345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:45.364945 7346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:45.364988 7348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:45.366175 7339 server_base.cc:1034] running on GCE node
I20260926 20:43:45.367208 7339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:45.369419 7339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:45.370828 7339 hybrid_clock.cc:648] HybridClock initialized: now 1790455425370658 us; error 107 us; skew 500 ppm
I20260926 20:43:45.371536 7339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:45 dist-test-slave-4d5t krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455425, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:45.375437 7339 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:43:45.377390 7339 webserver.cc:460] Webserver started at http://127.23.44.254:34431/ using document root <none> and password file <none>
I20260926 20:43:45.378194 7339 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:45.378302 7339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:45.384956 7339 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:43:45.388270 7355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:45.390121 7339 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:43:45.390322 7339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "0f9b50c7e5a44604a752c107c4dd51e9"
format_stamp: "Formatted at 2026-09-26 20:43:21 on dist-test-slave-4d5t"
server_key: "46681699cbdea15f6f2b04bafdcde52f"
server_key_iv: "954e30781421fadf4c9348e6110e7262"
server_key_version: "encryptionkey@0"
I20260926 20:43:45.391508 7339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:45.412259 7339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:45.417855 7339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:45.418169 7339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:45.442907 7339 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:44273
I20260926 20:43:45.442931 7406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:44273 every 8 connection(s)
I20260926 20:43:45.444839 7339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:43:45.451742 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 7339
I20260926 20:43:45.453101 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:46479
--local_ip_for_outbound_sockets=127.23.44.193
--tserver_master_addrs=127.23.44.254:44273
--webserver_port=42985
--webserver_interface=127.23.44.193
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:43:45.454823 7407 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:43:45.460495 7407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Bootstrap starting.
I20260926 20:43:45.484735 7407 log.cc:824] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Log is configured to *not* fsync() on all Append() calls
I20260926 20:43:45.500801 7407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:43:45.501492 7407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Bootstrap complete.
I20260926 20:43:45.512463 7407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } }
I20260926 20:43:45.513583 7407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f9b50c7e5a44604a752c107c4dd51e9, State: Initialized, Role: FOLLOWER
I20260926 20:43:45.514271 7407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } }
I20260926 20:43:45.514585 7407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:43:45.514744 7407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:43:45.514961 7407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 1 FOLLOWER]: Advancing to term 2
I20260926 20:43:45.519625 7407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } }
I20260926 20:43:45.520279 7407 leader_election.cc:302] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f9b50c7e5a44604a752c107c4dd51e9; no voters:
I20260926 20:43:45.521121 7407 leader_election.cc:288] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 20:43:45.521265 7411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 20:43:45.522467 7411 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 2 LEADER]: Becoming Leader. State: Replica: 0f9b50c7e5a44604a752c107c4dd51e9, State: Running, Role: LEADER
I20260926 20:43:45.523121 7411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } }
I20260926 20:43:45.523795 7407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:43:45.525832 7413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f9b50c7e5a44604a752c107c4dd51e9. Latest consensus state: current_term: 2 leader_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } } }
I20260926 20:43:45.526242 7413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: This master's current role is: LEADER
I20260926 20:43:45.526821 7412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } } }
I20260926 20:43:45.527133 7412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: This master's current role is: LEADER
I20260926 20:43:45.537003 7416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:43:45.547185 7416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:43:45.548975 7416 catalog_manager.cc:1259] Loaded cluster ID: 6fba881e86c049d88167e8830155778d
I20260926 20:43:45.549525 7416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:43:45.554411 7416 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:43:45.556434 7416 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Loaded TSK: 0
I20260926 20:43:45.601228 7416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Generated new TSK 1
I20260926 20:43:45.601713 7416 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 20:43:45.855453 7409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:45.855763 7409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:45.855916 7409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:45.863574 7409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:45.863699 7409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:45.863884 7409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:43:45.874114 7409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:46479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=42985
--tserver_master_addrs=127.23.44.254:44273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:45.875481 7409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:45.876902 7409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:45.893627 7434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:45.893554 7435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:45.894210 7437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:45.896026 7409 server_base.cc:1034] running on GCE node
I20260926 20:43:45.896929 7409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:45.898057 7409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:45.899344 7409 hybrid_clock.cc:648] HybridClock initialized: now 1790455425899267 us; error 69 us; skew 500 ppm
I20260926 20:43:45.899775 7409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:45 dist-test-slave-4d5t krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455425, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:45.904254 7409 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:43:45.906209 7409 webserver.cc:460] Webserver started at http://127.23.44.193:42985/ using document root <none> and password file <none>
I20260926 20:43:45.907019 7409 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:45.907181 7409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:45.913053 7409 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 20:43:45.916075 7444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:45.917667 7409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:43:45.917850 7409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "6412b3674fcc4595bd37f96961c9dd3a"
format_stamp: "Formatted at 2026-09-26 20:43:22 on dist-test-slave-4d5t"
server_key: "73a8634c323270bd7b1bfea9473368ec"
server_key_iv: "bf17692a74433d3a8ba7b3158f5ed294"
server_key_version: "encryptionkey@0"
I20260926 20:43:45.918853 7409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:45.940167 7409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:45.945086 7409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:45.945394 7409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:45.947304 7409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:43:45.949003 7409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:43:45.949141 7409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:45.949286 7409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:43:45.949393 7409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:45.992148 7409 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:46479
I20260926 20:43:45.992206 7556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:46479 every 8 connection(s)
I20260926 20:43:45.993683 7409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:43:45.994969 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 7409
I20260926 20:43:45.995669 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:35231
--local_ip_for_outbound_sockets=127.23.44.194
--tserver_master_addrs=127.23.44.254:44273
--webserver_port=42103
--webserver_interface=127.23.44.194
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:43:46 dist-test-slave-4d5t krb5kdc[6786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455425, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:43:46.024030 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:45.996922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:52903 (local address 127.23.44.254:44273)
0926 20:43:45.997875 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:45.997901 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:45.998575 (+ 674us) server_negotiation.cc:407] Connection header received
0926 20:43:46.001073 (+ 2498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:46.001111 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:46.001624 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:46.002046 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:46.003781 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:46.005704 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:46.008730 (+ 3026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:46.009373 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:46.009535 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:46.015047 (+ 5512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:46.015058 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:46.015075 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:46.015137 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:46.018335 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:46.019608 (+ 1273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:46.019616 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:46.019621 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:46.019806 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:46.020411 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:46.020414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:46.020416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:46.020978 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:46.021142 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:46.022440 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0926 20:43:46.022785 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":327,"threads_started":1}
I20260926 20:43:46.024415 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:45.997358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:52903)
0926 20:43:45.998343 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:45.998400 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:46.000337 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:46.002188 (+ 1851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:46.002196 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:46.002749 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:43:46.003495 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:46.003521 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:46.005942 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:46.005951 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:46.008496 (+ 2545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:46.008518 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:46.009652 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:46.009662 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:46.010023 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:46.010844 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:43:46.010883 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:43:46.014631 (+ 3748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:43:46.018544 (+ 3913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:46.018560 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:46.018581 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:46.019405 (+ 824us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:46.019965 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:46.019974 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:46.019982 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:46.020281 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:46.021180 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:43:46.021193 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:43:46.021720 (+ 527us) client_negotiation.cc:769] Sending connection context
0926 20:43:46.022115 (+ 395us) client_negotiation.cc:240] Negotiation successful
0926 20:43:46.022822 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":411,"threads_started":1}
I20260926 20:43:46.027872 7557 heartbeater.cc:347] Connected to a master server at 127.23.44.254:44273
I20260926 20:43:46.028276 7557 heartbeater.cc:464] Registering TS with master...
I20260926 20:43:46.029325 7557 heartbeater.cc:511] Master 127.23.44.254:44273 requested a full tablet report, sending...
I20260926 20:43:46.032356 7372 ts_manager.cc:194] Registered new tserver with Master: 6412b3674fcc4595bd37f96961c9dd3a (127.23.44.193:46479)
I20260926 20:43:46.034308 7372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:52903
W20260926 20:43:46.377665 7561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:46.377964 7561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:46.378126 7561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:46.385594 7561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:46.385722 7561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:46.385900 7561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:43:46.396010 7561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:35231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=42103
--tserver_master_addrs=127.23.44.254:44273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:46.397433 7561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:46.398720 7561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:46.412914 7569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:46.413051 7566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:46.412892 7567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:46.414578 7561 server_base.cc:1034] running on GCE node
I20260926 20:43:46.415355 7561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:46.416474 7561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:46.417776 7561 hybrid_clock.cc:648] HybridClock initialized: now 1790455426417721 us; error 41 us; skew 500 ppm
I20260926 20:43:46.418167 7561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:46 dist-test-slave-4d5t krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455426, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:46.421698 7561 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:43:46.423441 7561 webserver.cc:460] Webserver started at http://127.23.44.194:42103/ using document root <none> and password file <none>
I20260926 20:43:46.424235 7561 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:46.424398 7561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:46.430128 7561 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260926 20:43:46.433399 7576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:46.435003 7561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:43:46.435222 7561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "225cc05acc474d20927e9b25885d47f1"
format_stamp: "Formatted at 2026-09-26 20:43:22 on dist-test-slave-4d5t"
server_key: "71adafdb2034e3995aff77dfaca994f7"
server_key_iv: "d66c35bae1adac2cff1a7bea8f73d73e"
server_key_version: "encryptionkey@0"
I20260926 20:43:46.436328 7561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:46.463176 7561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:46.468381 7561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:46.468746 7561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:46.469817 7561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:43:46.471252 7561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:43:46.471378 7561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:46.471517 7561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:43:46.471611 7561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:46.511206 7561 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:35231
I20260926 20:43:46.511299 7688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:35231 every 8 connection(s)
I20260926 20:43:46.513146 7561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:43:46.519810 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 7561
I20260926 20:43:46.520484 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:41499
--local_ip_for_outbound_sockets=127.23.44.195
--tserver_master_addrs=127.23.44.254:44273
--webserver_port=42027
--webserver_interface=127.23.44.195
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 20:43:46 dist-test-slave-4d5t krb5kdc[6786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455426, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:43:46.537688 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:46.515898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:38385 (local address 127.23.44.254:44273)
0926 20:43:46.516095 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:46.516100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:46.517271 (+ 1171us) server_negotiation.cc:407] Connection header received
0926 20:43:46.519142 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:46.519147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:46.519271 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:46.519411 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:46.521189 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:46.521921 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:46.523632 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:46.524304 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:46.524516 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:46.529113 (+ 4597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:46.529119 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:46.529122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:46.529157 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:46.533119 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:46.534130 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:46.534138 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:46.534145 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:46.534300 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:46.534889 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:46.534898 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:46.534903 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:46.535336 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:46.535730 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:46.536524 (+ 794us) server_negotiation.cc:299] Negotiation successful
0926 20:43:46.536958 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:43:46.538335 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:46.516335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:38385)
0926 20:43:46.517105 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:46.517142 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:46.518855 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:46.519669 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:46.519677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:46.520274 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:43:46.521006 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:46.521025 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:46.522092 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:46.522103 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:46.523457 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:46.523471 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:46.524677 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:46.524709 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:46.525106 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:46.525952 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:43:46.525994 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:43:46.528845 (+ 2851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:43:46.533341 (+ 4496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:46.533353 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:46.533376 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:46.533962 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:46.534459 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:46.534463 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:46.534465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:46.534672 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:46.535538 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:43:46.535545 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:43:46.535975 (+ 430us) client_negotiation.cc:769] Sending connection context
0926 20:43:46.536276 (+ 301us) client_negotiation.cc:240] Negotiation successful
0926 20:43:46.536871 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":332,"threads_started":1}
I20260926 20:43:46.540450 7689 heartbeater.cc:347] Connected to a master server at 127.23.44.254:44273
I20260926 20:43:46.540966 7689 heartbeater.cc:464] Registering TS with master...
I20260926 20:43:46.542246 7689 heartbeater.cc:511] Master 127.23.44.254:44273 requested a full tablet report, sending...
I20260926 20:43:46.544509 7372 ts_manager.cc:194] Registered new tserver with Master: 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231)
I20260926 20:43:46.545457 7372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:38385
W20260926 20:43:46.879233 7692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:43:46.879552 7692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:43:46.879694 7692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:43:46.887451 7692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:43:46.887571 7692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:43:46.887727 7692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:43:46.898031 7692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:41499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=42027
--tserver_master_addrs=127.23.44.254:44273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:43:46.899374 7692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:43:46.900750 7692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:43:46.914682 7698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:46.914721 7700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:43:46.914645 7697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:43:46.915729 7692 server_base.cc:1034] running on GCE node
I20260926 20:43:46.916400 7692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:43:46.917411 7692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:43:46.918676 7692 hybrid_clock.cc:648] HybridClock initialized: now 1790455426918620 us; error 46 us; skew 500 ppm
I20260926 20:43:46.919030 7692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:43:46 dist-test-slave-4d5t krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455426, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:43:46.922485 7692 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:43:46.924225 7692 webserver.cc:460] Webserver started at http://127.23.44.195:42027/ using document root <none> and password file <none>
I20260926 20:43:46.925065 7692 fs_manager.cc:359] Metadata directory not provided
I20260926 20:43:46.925210 7692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:43:46.930874 7692 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 20:43:46.933878 7707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:46.935310 7692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 20:43:46.935498 7692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "2e01cf8bdc2f4717b7a156574501b253"
format_stamp: "Formatted at 2026-09-26 20:43:23 on dist-test-slave-4d5t"
server_key: "339550196f560b5757b900dc042bd51c"
server_key_iv: "25000752c24ecf76486c33960b210fdc"
server_key_version: "encryptionkey@0"
I20260926 20:43:46.936486 7692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:43:46.972529 7692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:43:46.977243 7692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:43:46.977481 7692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:43:46.978482 7692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:43:46.979818 7692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:43:46.979904 7692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:46.979983 7692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:43:46.980038 7692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:43:47.017138 7692 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:41499
I20260926 20:43:47.017246 7819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:41499 every 8 connection(s)
I20260926 20:43:47.018707 7692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:43:47.024788 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 7692
Sep 26 20:43:47 dist-test-slave-4d5t krb5kdc[6786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455426, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:43:47.039552 7557 heartbeater.cc:503] Master 127.23.44.254:44273 was elected leader, sending a full tablet report...
I20260926 20:43:47.047024 7824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.025963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43686 (local address 127.23.44.254:44273)
0926 20:43:47.027111 (+ 1148us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:47.027116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:47.027136 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:43:47.027185 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:47.027190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:47.027262 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:47.027382 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:47.029682 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.030808 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.032961 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.033624 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.033758 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.037160 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:47.037167 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:47.037172 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:47.037215 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:47.042289 (+ 5074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:47.043399 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:47.043406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:47.043409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:47.043810 (+ 401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:47.044598 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:47.044604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:47.044606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:47.045524 (+ 918us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:47.045839 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:47.046307 (+ 468us) server_negotiation.cc:299] Negotiation successful
0926 20:43:47.046532 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":1011,"spinlock_wait_cycles":6144,"thread_start_us":863,"threads_started":1}
I20260926 20:43:47.048780 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.021656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:59329 (local address 127.23.44.254:44273)
0926 20:43:47.021870 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:47.021875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:47.023111 (+ 1236us) server_negotiation.cc:407] Connection header received
0926 20:43:47.025498 (+ 2387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:47.025505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:47.025640 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:47.025757 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:47.028119 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.029491 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.031636 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.032286 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.032431 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.037772 (+ 5341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:47.037777 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:47.037781 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:47.037825 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:47.042575 (+ 4750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:47.043541 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:47.043550 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:47.043552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:47.043810 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:47.044576 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:47.044582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:47.044586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:47.045524 (+ 938us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:47.045730 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:47.047580 (+ 1850us) server_negotiation.cc:299] Negotiation successful
0926 20:43:47.048387 (+ 807us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":86,"spinlock_wait_cycles":23808}
I20260926 20:43:47.049928 7822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.022189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:59329)
0926 20:43:47.022924 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.022968 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:47.025154 (+ 2186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:47.026215 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:47.026228 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:47.027116 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:43:47.027919 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:47.027939 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.029713 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.029720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:47.031465 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:47.031489 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.032643 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.032663 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:47.033299 (+ 636us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.034200 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:43:47.034224 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:43:47.037523 (+ 3299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:43:47.042726 (+ 5203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:47.042736 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:47.042763 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:47.043395 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:47.044041 (+ 646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:43:47.044047 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:43:47.044052 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:43:47.044391 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:43:47.045716 (+ 1325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:43:47.045726 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:43:47.046286 (+ 560us) client_negotiation.cc:769] Sending connection context
0926 20:43:47.046961 (+ 675us) client_negotiation.cc:240] Negotiation successful
0926 20:43:47.047875 (+ 914us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":288,"threads_started":1}
I20260926 20:43:47.051754 7820 heartbeater.cc:347] Connected to a master server at 127.23.44.254:44273
I20260926 20:43:47.052188 7820 heartbeater.cc:464] Registering TS with master...
I20260926 20:43:47.053253 7820 heartbeater.cc:511] Master 127.23.44.254:44273 requested a full tablet report, sending...
I20260926 20:43:47.055459 7372 ts_manager.cc:194] Registered new tserver with Master: 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499)
I20260926 20:43:47.056631 7372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:59329
I20260926 20:43:47.063290 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:43:47.070727 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.064397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43698 (local address 127.23.44.254:44273)
0926 20:43:47.064685 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:47.064696 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:47.065166 (+ 470us) server_negotiation.cc:407] Connection header received
0926 20:43:47.065406 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:47.065412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:47.065575 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:47.065734 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:43:47.066152 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.066991 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.068736 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.069138 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.069248 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.069983 (+ 735us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:43:47.070220 (+ 237us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 20:43:47.070533 (+ 313us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43698: token expired
Metrics: {"server-negotiator.queue_time_us":83}
W20260926 20:43:47.070914 7559 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43698: token expired
W20260926 20:43:47.071717 7293 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.23.44.254:44273)
I20260926 20:43:47.087113 7559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.072207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43700 (local address 127.23.44.254:44273)
0926 20:43:47.072403 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:47.072409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:47.072428 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:43:47.072633 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:47.072638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:47.072886 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:47.073097 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:47.075026 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.075955 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.077731 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.078218 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.078401 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.081426 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:47.081431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:47.081435 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:47.081473 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:47.084109 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:47.085042 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:47.085047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:47.085050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:47.085173 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:47.085566 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:47.085570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:47.085573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:47.085907 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:47.086168 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:47.086512 (+ 344us) server_negotiation.cc:299] Negotiation successful
0926 20:43:47.086759 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 20:43:47.091564 7372 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:43:47.100237 7372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:43:47.140018 7830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.114185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:41499 (local address 127.0.0.1:44766)
0926 20:43:47.128417 (+ 14232us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.128437 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:47.128581 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:47.130133 (+ 1552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:47.130145 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:47.130189 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:47.130985 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:47.131036 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.133634 (+ 2598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.133640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:47.136348 (+ 2708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:47.136364 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.138806 (+ 2442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.138813 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:47.139030 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.139084 (+ 54us) client_negotiation.cc:769] Sending connection context
0926 20:43:47.139252 (+ 168us) client_negotiation.cc:240] Negotiation successful
0926 20:43:47.139489 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14093,"mutex_wait_us":3,"thread_start_us":14562,"threads_started":1}
I20260926 20:43:47.140599 7833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.114785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44766 (local address 127.23.44.195:41499)
0926 20:43:47.127649 (+ 12864us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:47.127663 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:47.128807 (+ 1144us) server_negotiation.cc:407] Connection header received
0926 20:43:47.128953 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:47.128962 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:47.129346 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:47.129576 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:47.131491 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.133452 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.136601 (+ 3149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.138513 (+ 1912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.138680 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.139002 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:47.139344 (+ 342us) server_negotiation.cc:299] Negotiation successful
0926 20:43:47.139568 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12551,"thread_start_us":12905,"threads_started":1}
I20260926 20:43:47.142601 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.130087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43554 (local address 127.23.44.193:46479)
0926 20:43:47.133976 (+ 3889us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:47.133987 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:47.134022 (+ 35us) server_negotiation.cc:407] Connection header received
0926 20:43:47.134133 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:47.134143 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:47.134446 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:47.135371 (+ 925us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:47.135629 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.137022 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.139457 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.141240 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.141393 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.141634 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:47.142068 (+ 434us) server_negotiation.cc:299] Negotiation successful
0926 20:43:47.142247 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3667,"thread_start_us":2341,"threads_started":1}
I20260926 20:43:47.142899 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.130914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:46479 (local address 127.0.0.1:43554)
0926 20:43:47.133221 (+ 2307us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.133244 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:47.133387 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:47.134713 (+ 1326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:47.134718 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:47.134759 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:47.135160 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:47.135175 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.137161 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.137169 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:47.139271 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:47.139283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.141530 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.141538 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:47.141816 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.141871 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 20:43:47.142003 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 20:43:47.142191 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2122,"mutex_wait_us":30,"thread_start_us":2575,"threads_started":1}
I20260926 20:43:47.145758 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.114012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:35231 (local address 127.0.0.1:37524)
0926 20:43:47.128013 (+ 14001us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.128096 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:47.128464 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:47.135465 (+ 7001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:47.135470 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:47.135495 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:47.135912 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:47.135922 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.139002 (+ 3080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.139009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:47.141815 (+ 2806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:47.141832 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.144829 (+ 2997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.144836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:47.145023 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.145056 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 20:43:47.145169 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 20:43:47.145320 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13615,"mutex_wait_us":28,"thread_start_us":14149,"threads_started":1}
I20260926 20:43:47.145763 7832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.114217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37524 (local address 127.23.44.194:35231)
0926 20:43:47.134978 (+ 20761us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:47.134985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:47.135010 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:43:47.135084 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:47.135089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:47.135265 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:47.135472 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:47.136087 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.138799 (+ 2712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.143213 (+ 4414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.144493 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.144760 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.144966 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:47.145225 (+ 259us) server_negotiation.cc:299] Negotiation successful
0926 20:43:47.145441 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20586,"thread_start_us":20443,"threads_started":1}
I20260926 20:43:47.145541 7755 tablet_service.cc:1467] Processing CreateTablet for tablet a92ad71b81a64552be31c4c4f66f9363 (DEFAULT_TABLE table=test-table [id=2030f98d682a41f8bdce65f5dbe07ffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:43:47.147600 7755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a92ad71b81a64552be31c4c4f66f9363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:43:47.147713 7492 tablet_service.cc:1467] Processing CreateTablet for tablet a92ad71b81a64552be31c4c4f66f9363 (DEFAULT_TABLE table=test-table [id=2030f98d682a41f8bdce65f5dbe07ffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:43:47.149255 7492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a92ad71b81a64552be31c4c4f66f9363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:43:47.149452 7624 tablet_service.cc:1467] Processing CreateTablet for tablet a92ad71b81a64552be31c4c4f66f9363 (DEFAULT_TABLE table=test-table [id=2030f98d682a41f8bdce65f5dbe07ffc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:43:47.150496 7624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a92ad71b81a64552be31c4c4f66f9363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:43:47.162919 7836 tablet_bootstrap.cc:492] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1: Bootstrap starting.
I20260926 20:43:47.167335 7836 tablet_bootstrap.cc:654] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1: Neither blocks nor log segments found. Creating new log.
I20260926 20:43:47.169050 7836 log.cc:824] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1: Log is configured to *not* fsync() on all Append() calls
I20260926 20:43:47.173849 7837 tablet_bootstrap.cc:492] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253: Bootstrap starting.
I20260926 20:43:47.176328 7838 tablet_bootstrap.cc:492] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: Bootstrap starting.
I20260926 20:43:47.178088 7836 tablet_bootstrap.cc:492] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1: No bootstrap required, opened a new log
I20260926 20:43:47.178421 7836 ts_tablet_manager.cc:1397] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.007s
I20260926 20:43:47.179363 7837 tablet_bootstrap.cc:654] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253: Neither blocks nor log segments found. Creating new log.
I20260926 20:43:47.179952 7838 tablet_bootstrap.cc:654] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: Neither blocks nor log segments found. Creating new log.
I20260926 20:43:47.180747 7837 log.cc:824] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253: Log is configured to *not* fsync() on all Append() calls
I20260926 20:43:47.181736 7838 log.cc:824] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: Log is configured to *not* fsync() on all Append() calls
I20260926 20:43:47.187558 7838 tablet_bootstrap.cc:492] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: No bootstrap required, opened a new log
I20260926 20:43:47.187654 7837 tablet_bootstrap.cc:492] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253: No bootstrap required, opened a new log
I20260926 20:43:47.187925 7838 ts_tablet_manager.cc:1397] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260926 20:43:47.187919 7837 ts_tablet_manager.cc:1397] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.006s
I20260926 20:43:47.190788 7836 raft_consensus.cc:348] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:47.191280 7836 raft_consensus.cc:374] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:43:47.191377 7836 raft_consensus.cc:729] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 225cc05acc474d20927e9b25885d47f1, State: Initialized, Role: FOLLOWER
I20260926 20:43:47.192060 7836 consensus_queue.cc:260] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:47.193406 7689 heartbeater.cc:503] Master 127.23.44.254:44273 was elected leader, sending a full tablet report...
I20260926 20:43:47.194087 7836 ts_tablet_manager.cc:1428] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1: Time spent starting tablet: real 0.016s user 0.012s sys 0.003s
I20260926 20:43:47.198053 7837 raft_consensus.cc:348] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:47.198556 7837 raft_consensus.cc:374] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:43:47.198668 7837 raft_consensus.cc:729] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e01cf8bdc2f4717b7a156574501b253, State: Initialized, Role: FOLLOWER
I20260926 20:43:47.198458 7838 raft_consensus.cc:348] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:47.198993 7838 raft_consensus.cc:374] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:43:47.199126 7838 raft_consensus.cc:729] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6412b3674fcc4595bd37f96961c9dd3a, State: Initialized, Role: FOLLOWER
I20260926 20:43:47.199458 7837 consensus_queue.cc:260] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:47.199754 7838 consensus_queue.cc:260] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:47.200735 7820 heartbeater.cc:503] Master 127.23.44.254:44273 was elected leader, sending a full tablet report...
I20260926 20:43:47.201422 7837 ts_tablet_manager.cc:1428] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260926 20:43:47.202297 7838 ts_tablet_manager.cc:1428] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: Time spent starting tablet: real 0.014s user 0.007s sys 0.008s
I20260926 20:43:47.202791 7842 raft_consensus.cc:482] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:43:47.203078 7842 raft_consensus.cc:504] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:47.205137 7842 leader_election.cc:288] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499), 6412b3674fcc4595bd37f96961c9dd3a (127.23.44.193:46479)
I20260926 20:43:47.213996 7833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.205485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:52823 (local address 127.23.44.195:41499)
0926 20:43:47.205690 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:47.205697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:47.207124 (+ 1427us) server_negotiation.cc:407] Connection header received
0926 20:43:47.207225 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:47.207231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:47.207339 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:47.207446 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:47.208432 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.209336 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.211998 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.212925 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.213126 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.213213 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:47.213590 (+ 377us) server_negotiation.cc:299] Negotiation successful
0926 20:43:47.213726 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:43:47.213999 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.207347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:46443 (local address 127.23.44.193:46479)
0926 20:43:47.207731 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:47.207737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:47.207857 (+ 120us) server_negotiation.cc:407] Connection header received
0926 20:43:47.208015 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:47.208019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:47.208076 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:47.208154 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:43:47.208888 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.209614 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.212063 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.212924 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.213115 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.213198 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:47.213617 (+ 419us) server_negotiation.cc:299] Negotiation successful
0926 20:43:47.213747 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233}
I20260926 20:43:47.214178 7845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.205576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:41499 (local address 127.23.44.194:52823)
0926 20:43:47.206901 (+ 1325us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.206921 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:47.207103 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:47.207808 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:47.207815 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:47.207842 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:47.208238 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:47.208247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.209491 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.209494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:47.211843 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:47.211864 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.213212 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.213219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:47.213414 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.213452 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 20:43:47.213564 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 20:43:47.213739 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1176,"mutex_wait_us":69,"thread_start_us":980,"threads_started":1}
I20260926 20:43:47.214257 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.207109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:46479 (local address 127.23.44.194:46443)
0926 20:43:47.207735 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.207753 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 20:43:47.207875 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:43:47.208205 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:43:47.208210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:43:47.208239 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:43:47.208725 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:47.208740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.209777 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.209786 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:47.211937 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:43:47.211951 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.213172 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.213180 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:43:47.213427 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.213469 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 20:43:47.213577 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 20:43:47.213732 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":80,"thread_start_us":292,"threads_started":1}
I20260926 20:43:47.215293 7512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a92ad71b81a64552be31c4c4f66f9363" candidate_uuid: "225cc05acc474d20927e9b25885d47f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6412b3674fcc4595bd37f96961c9dd3a" is_pre_election: true
I20260926 20:43:47.215415 7775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a92ad71b81a64552be31c4c4f66f9363" candidate_uuid: "225cc05acc474d20927e9b25885d47f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e01cf8bdc2f4717b7a156574501b253" is_pre_election: true
I20260926 20:43:47.215852 7512 raft_consensus.cc:2454] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 225cc05acc474d20927e9b25885d47f1 in term 0.
I20260926 20:43:47.215919 7775 raft_consensus.cc:2454] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 225cc05acc474d20927e9b25885d47f1 in term 0.
I20260926 20:43:47.216657 7579 leader_election.cc:302] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 225cc05acc474d20927e9b25885d47f1, 6412b3674fcc4595bd37f96961c9dd3a; no voters:
I20260926 20:43:47.217023 7842 raft_consensus.cc:2781] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:43:47.217187 7842 raft_consensus.cc:482] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:43:47.217319 7842 raft_consensus.cc:3037] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:43:47.219561 7842 raft_consensus.cc:504] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:47.220165 7842 leader_election.cc:288] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [CANDIDATE]: Term 1 election: Requested vote from peers 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499), 6412b3674fcc4595bd37f96961c9dd3a (127.23.44.193:46479)
I20260926 20:43:47.220968 7512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a92ad71b81a64552be31c4c4f66f9363" candidate_uuid: "225cc05acc474d20927e9b25885d47f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6412b3674fcc4595bd37f96961c9dd3a"
I20260926 20:43:47.221204 7512 raft_consensus.cc:3037] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:43:47.221038 7775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a92ad71b81a64552be31c4c4f66f9363" candidate_uuid: "225cc05acc474d20927e9b25885d47f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e01cf8bdc2f4717b7a156574501b253"
I20260926 20:43:47.221537 7775 raft_consensus.cc:3037] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:43:47.224157 7775 raft_consensus.cc:2454] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 225cc05acc474d20927e9b25885d47f1 in term 1.
I20260926 20:43:47.224255 7512 raft_consensus.cc:2454] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 225cc05acc474d20927e9b25885d47f1 in term 1.
I20260926 20:43:47.224717 7579 leader_election.cc:302] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 225cc05acc474d20927e9b25885d47f1, 2e01cf8bdc2f4717b7a156574501b253; no voters:
I20260926 20:43:47.225077 7842 raft_consensus.cc:2781] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:43:47.225605 7842 raft_consensus.cc:686] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [term 1 LEADER]: Becoming Leader. State: Replica: 225cc05acc474d20927e9b25885d47f1, State: Running, Role: LEADER
I20260926 20:43:47.226097 7842 consensus_queue.cc:237] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:43:47.230804 7371 catalog_manager.cc:5495] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 225cc05acc474d20927e9b25885d47f1 (127.23.44.194). New cstate: current_term: 1 leader_uuid: "225cc05acc474d20927e9b25885d47f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } health_report { overall_health: UNKNOWN } } }
W20260926 20:43:47.246196 7558 tablet.cc:2369] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:43:47.265543 7690 tablet.cc:2369] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:43:47.270627 7821 tablet.cc:2369] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:43:47.272904 7832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.265156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37528 (local address 127.23.44.194:35231)
0926 20:43:47.265403 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:47.265411 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:47.265749 (+ 338us) server_negotiation.cc:407] Connection header received
0926 20:43:47.266085 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:47.266090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:47.266229 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:47.266396 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:43:47.267036 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.267810 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.269485 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.269966 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.270086 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.271154 (+ 1068us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:43:47.272001 (+ 847us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:43:47.272170 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:47.272409 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 20:43:47.272519 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:43:47.283686 7775 raft_consensus.cc:1261] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253 [term 1 FOLLOWER]: Refusing update from remote peer 225cc05acc474d20927e9b25885d47f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:43:47.284389 7512 raft_consensus.cc:1261] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Refusing update from remote peer 225cc05acc474d20927e9b25885d47f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:43:47.284924 7847 consensus_queue.cc:1035] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:43:47.285383 7842 consensus_queue.cc:1035] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:43:47.313978 7371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:43:47.314329 7371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:43:47.318898 7371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Sending DeleteTablet for 3 replicas of tablet a92ad71b81a64552be31c4c4f66f9363
I20260926 20:43:47.320389 7624 tablet_service.cc:1514] Processing DeleteTablet for tablet a92ad71b81a64552be31c4c4f66f9363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:43:47 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:37524
I20260926 20:43:47.320384 7755 tablet_service.cc:1514] Processing DeleteTablet for tablet a92ad71b81a64552be31c4c4f66f9363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:43:47 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:44766
I20260926 20:43:47.320626 7492 tablet_service.cc:1514] Processing DeleteTablet for tablet a92ad71b81a64552be31c4c4f66f9363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:43:47 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:43554
I20260926 20:43:47.322055 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 7409
I20260926 20:43:47.334339 7861 tablet_replica.cc:330] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1: stopping tablet replica
I20260926 20:43:47.335325 7861 raft_consensus.cc:2229] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:43:47.335752 7860 tablet_replica.cc:330] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253: stopping tablet replica
I20260926 20:43:47.336192 7860 raft_consensus.cc:2229] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:43:47.336454 7861 raft_consensus.cc:2258] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:43:47.336767 7860 raft_consensus.cc:2258] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:43:47.341480 7861 ts_tablet_manager.cc:1905] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:43:47.346760 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.323001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43556 (local address 127.23.44.193:46479)
0926 20:43:47.323216 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:47.323224 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:47.323430 (+ 206us) server_negotiation.cc:407] Connection header received
0926 20:43:47.323638 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:47.323644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:47.323739 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:47.323890 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:47.326680 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.328291 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.331239 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.331982 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.332161 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.337555 (+ 5394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:47.337580 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:47.337598 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:47.337661 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:47.342206 (+ 4545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:47.343257 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:47.343267 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:47.343271 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:47.343459 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:47.343929 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:47.343935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:47.343940 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:47.344437 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:47.344796 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:47.346060 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0926 20:43:47.346323 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 20:43:47.347481 7552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:47.349227 7860 ts_tablet_manager.cc:1905] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:43:47.366812 7861 ts_tablet_manager.cc:1918] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:43:47.367084 7861 log.cc:1196] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/a92ad71b81a64552be31c4c4f66f9363
I20260926 20:43:47.367841 7861 ts_tablet_manager.cc:1939] T a92ad71b81a64552be31c4c4f66f9363 P 225cc05acc474d20927e9b25885d47f1: Deleting consensus metadata
I20260926 20:43:47.370241 7860 ts_tablet_manager.cc:1918] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:43:47.370448 7860 log.cc:1196] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/a92ad71b81a64552be31c4c4f66f9363
I20260926 20:43:47.371054 7860 ts_tablet_manager.cc:1939] T a92ad71b81a64552be31c4c4f66f9363 P 2e01cf8bdc2f4717b7a156574501b253: Deleting consensus metadata
I20260926 20:43:47.372385 7359 catalog_manager.cc:4841] TS 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231): tablet a92ad71b81a64552be31c4c4f66f9363 (table test-table [id=2030f98d682a41f8bdce65f5dbe07ffc]) successfully deleted
I20260926 20:43:47.373271 7358 catalog_manager.cc:4841] TS 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499): tablet a92ad71b81a64552be31c4c4f66f9363 (table test-table [id=2030f98d682a41f8bdce65f5dbe07ffc]) successfully deleted
I20260926 20:43:47.456907 7862 tablet_replica.cc:330] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: stopping tablet replica
I20260926 20:43:47.457201 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 7409
I20260926 20:43:47.477761 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 7561
W20260926 20:43:47.477797 7358 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.44.193:46479 (error 108)
I20260926 20:43:47.480299 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.479771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:46479 (local address 127.0.0.1:43562)
0926 20:43:47.480053 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.480147 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:46479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
W20260926 20:43:47.480706 7358 catalog_manager.cc:4584] TS 6412b3674fcc4595bd37f96961c9dd3a (127.23.44.193:46479): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a92ad71b81a64552be31c4c4f66f9363: Network error: Client connection negotiation failed: client connection to 127.23.44.193:46479: connect: Connection refused (error 111)
I20260926 20:43:47.501993 7832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.478785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37536 (local address 127.23.44.194:35231)
0926 20:43:47.479018 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:47.479026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:47.479385 (+ 359us) server_negotiation.cc:407] Connection header received
0926 20:43:47.479477 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:47.479481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:47.479605 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:47.479744 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:47.481619 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.482408 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.484389 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.484966 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.485080 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.493750 (+ 8670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:47.493768 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:47.493783 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:47.493833 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:47.498207 (+ 4374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:47.499164 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:47.499172 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:47.499175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:47.499370 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:47.499849 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:47.499854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:47.499856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:47.500309 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:47.500719 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:47.501177 (+ 458us) server_negotiation.cc:299] Negotiation successful
0926 20:43:47.501522 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 20:43:47.502895 7684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:47.534971 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.534428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:46479 (local address 127.0.0.1:43572)
0926 20:43:47.534674 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.534808 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:46479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260926 20:43:47.536036 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.535690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:46479 (local address 127.0.0.1:43586)
0926 20:43:47.535841 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.535930 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:46479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 20:43:47.596725 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.596060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:46479 (local address 127.0.0.1:43598)
0926 20:43:47.596375 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.596501 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:46479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
I20260926 20:43:47.597781 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.597444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:46479 (local address 127.0.0.1:43600)
0926 20:43:47.597597 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.597676 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:46479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 20:43:47.621793 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 7561
I20260926 20:43:47.642544 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 7692
I20260926 20:43:47.658886 7833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.643529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44770 (local address 127.23.44.195:41499)
0926 20:43:47.643722 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 20:43:47.643727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:43:47.643746 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:43:47.643889 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:43:47.643892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:43:47.644004 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:43:47.644091 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:43:47.645728 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.646603 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.648345 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:43:47.648983 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:43:47.649100 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:43:47.651711 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:43:47.651726 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:43:47.651743 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:43:47.651789 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:43:47.655168 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:47.655971 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:47.655979 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:47.655982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:47.656142 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:43:47.656786 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:43:47.656792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:43:47.656795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:43:47.657388 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:43:47.657842 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0926 20:43:47.658256 (+ 414us) server_negotiation.cc:299] Negotiation successful
0926 20:43:47.658532 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:43:47.659686 7815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:47.678222 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.677681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:46479 (local address 127.0.0.1:43608)
0926 20:43:47.677913 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.678024 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:46479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 20:43:47.679365 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.678931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:46479 (local address 127.0.0.1:43622)
0926 20:43:47.679147 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.679234 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:46479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260926 20:43:47.755640 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 7692
I20260926 20:43:47.773043 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 7339
I20260926 20:43:47.774333 7402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:43:47.890266 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:43:47.889627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:46479 (local address 127.0.0.1:43634)
0926 20:43:47.889921 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0926 20:43:47.890050 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:46479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260926 20:43:47.890376 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 7339
I20260926 20:44:07.914885 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:44273
--webserver_interface=127.23.44.254
--webserver_port=34431
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:44273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:08.281816 7868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:08.282132 7868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:08.282225 7868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:08.289702 7868 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 20:44:08.289811 7868 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 20:44:08.289849 7868 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 20:44:08.289894 7868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:44:08.289949 7868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:08.290019 7868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:08.290063 7868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:44:08.290122 7868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:44:08.300251 7868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:44273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:44273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=34431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:08.301656 7868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:08.303066 7868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:08.317227 7873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:08.317220 7874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:08.317220 7876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:08.317979 7868 server_base.cc:1034] running on GCE node
I20260926 20:44:08.318868 7868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:08.320637 7868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:08.322183 7868 hybrid_clock.cc:648] HybridClock initialized: now 1790455448322041 us; error 90 us; skew 500 ppm
I20260926 20:44:08.322659 7868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:08 dist-test-slave-4d5t krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455448, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:08.327287 7868 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:44:08.329339 7868 webserver.cc:460] Webserver started at http://127.23.44.254:34431/ using document root <none> and password file <none>
I20260926 20:44:08.330112 7868 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:08.330215 7868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:08.336282 7868 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 20:44:08.339666 7883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:08.341758 7868 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:44:08.341990 7868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "0f9b50c7e5a44604a752c107c4dd51e9"
format_stamp: "Formatted at 2026-09-26 20:43:21 on dist-test-slave-4d5t"
server_key: "46681699cbdea15f6f2b04bafdcde52f"
server_key_iv: "954e30781421fadf4c9348e6110e7262"
server_key_version: "encryptionkey@0"
I20260926 20:44:08.343070 7868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:08.370682 7868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:08.375053 7868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:08.375391 7868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:08.395282 7868 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:44273
I20260926 20:44:08.395326 7934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:44273 every 8 connection(s)
I20260926 20:44:08.396864 7868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:44:08.405292 7935 sys_catalog.cc:263] Verifying existing consensus state
I20260926 20:44:08.405292 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 7868
I20260926 20:44:08.406442 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:46479
--local_ip_for_outbound_sockets=127.23.44.193
--tserver_master_addrs=127.23.44.254:44273
--webserver_port=42985
--webserver_interface=127.23.44.193
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:44:08.411303 7935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Bootstrap starting.
I20260926 20:44:08.434984 7935 log.cc:824] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:08.453311 7935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:44:08.453918 7935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Bootstrap complete.
I20260926 20:44:08.463210 7935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } }
I20260926 20:44:08.464174 7935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f9b50c7e5a44604a752c107c4dd51e9, State: Initialized, Role: FOLLOWER
I20260926 20:44:08.464884 7935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } }
I20260926 20:44:08.465137 7935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:44:08.465270 7935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:44:08.465437 7935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 2 FOLLOWER]: Advancing to term 3
I20260926 20:44:08.470672 7935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } }
I20260926 20:44:08.471217 7935 leader_election.cc:302] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f9b50c7e5a44604a752c107c4dd51e9; no voters:
I20260926 20:44:08.471943 7935 leader_election.cc:288] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 20:44:08.472093 7939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 20:44:08.473423 7939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [term 3 LEADER]: Becoming Leader. State: Replica: 0f9b50c7e5a44604a752c107c4dd51e9, State: Running, Role: LEADER
I20260926 20:44:08.474079 7939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } }
I20260926 20:44:08.474278 7935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:44:08.476775 7941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f9b50c7e5a44604a752c107c4dd51e9. Latest consensus state: current_term: 3 leader_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } } }
I20260926 20:44:08.477247 7941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:08.477674 7940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f9b50c7e5a44604a752c107c4dd51e9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44273 } } }
I20260926 20:44:08.478107 7940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9 [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:08.484907 7947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:44:08.494148 7947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:44:08.496296 7947 catalog_manager.cc:1259] Loaded cluster ID: 6fba881e86c049d88167e8830155778d
I20260926 20:44:08.496527 7947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:44:08.501071 7956 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 20:44:08.501982 7947 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:44:08.503500 7947 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Loaded TSK: 0
I20260926 20:44:08.503647 7947 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0f9b50c7e5a44604a752c107c4dd51e9: Loaded TSK: 1
F20260926 20:44:08.504699 7947 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1ae64239ad google::LogMessage::Fail() at ??:0
@ 0x7f1ae642688a google::LogMessage::SendToLog() at ??:0
@ 0x7f1ae64234de google::LogMessage::Flush() at ??:0
@ 0x7f1ae6424a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1aea1877d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f1aea17673e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f1afbe561c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f1afbe55783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f1afbee12a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f1afbee10c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f1afbf2b5e9 std::function<>::operator()() at ??:0
@ 0x7f1afbe59cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f1afbe57c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1afbedf610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f1afbedf439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f1ae906f345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1ae907291c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f1ae9072799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f1ae904deaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1aec3276db start_thread at ??:0
@ 0x7f1ae52bd61f clone at ??:0
W20260926 20:44:08.824275 7937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:08.824654 7937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:08.824769 7937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:08.832280 7937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:08.832419 7937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:08.832540 7937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:44:08.843735 7937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:46479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=42985
--tserver_master_addrs=127.23.44.254:44273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:08.846187 7937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:08.847959 7937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:08.864328 7983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:08.864351 7982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:08.864282 7985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:08.865342 7937 server_base.cc:1034] running on GCE node
I20260926 20:44:08.866039 7937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:08.867201 7937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:08.868489 7937 hybrid_clock.cc:648] HybridClock initialized: now 1790455448868430 us; error 65 us; skew 500 ppm
I20260926 20:44:08.868928 7937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:08 dist-test-slave-4d5t krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455448, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:08.872439 7937 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:44:08.874545 7937 webserver.cc:460] Webserver started at http://127.23.44.193:42985/ using document root <none> and password file <none>
I20260926 20:44:08.875304 7937 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:08.875459 7937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:08.881666 7937 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:44:08.885735 7992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:08.887802 7937 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:44:08.888054 7937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "6412b3674fcc4595bd37f96961c9dd3a"
format_stamp: "Formatted at 2026-09-26 20:43:22 on dist-test-slave-4d5t"
server_key: "73a8634c323270bd7b1bfea9473368ec"
server_key_iv: "bf17692a74433d3a8ba7b3158f5ed294"
server_key_version: "encryptionkey@0"
I20260926 20:44:08.889393 7937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:08.904503 7937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:08.909567 7937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:08.909906 7937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:08.911023 7937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:08.913002 7999 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260926 20:44:08.919284 7937 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 20:44:08.919479 7937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260926 20:44:08.919653 7937 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 20:44:08.923305 7937 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 20:44:08.923444 7937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20260926 20:44:08.924166 7999 tablet_bootstrap.cc:492] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: Bootstrap starting.
I20260926 20:44:08.957074 7999 log.cc:824] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:08.970598 7999 tablet_bootstrap.cc:492] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 20:44:08.971240 7999 tablet_bootstrap.cc:492] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: Bootstrap complete.
I20260926 20:44:08.972776 7999 ts_tablet_manager.cc:1397] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: Time spent bootstrapping tablet: real 0.049s user 0.034s sys 0.014s
I20260926 20:44:08.981932 7937 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:46479
I20260926 20:44:08.981969 8106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:46479 every 8 connection(s)
I20260926 20:44:08.983424 7937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:44:08.982527 7999 raft_consensus.cc:348] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:44:08.983633 7999 raft_consensus.cc:729] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6412b3674fcc4595bd37f96961c9dd3a, State: Initialized, Role: FOLLOWER
I20260926 20:44:08.984493 7999 consensus_queue.cc:260] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:44:08.986598 7999 ts_tablet_manager.cc:1428] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260926 20:44:08.989091 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 7937
I20260926 20:44:08.989148 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:08.986534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:57009)
0926 20:44:08.987520 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:08.987833 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":474,"threads_started":1}
W20260926 20:44:08.989938 8107 heartbeater.cc:644] Failed to heartbeat to 127.23.44.254:44273 (0 consecutive failures): Network error: Failed to ping master at 127.23.44.254:44273: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
I20260926 20:44:08.989967 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:35231
--local_ip_for_outbound_sockets=127.23.44.194
--tserver_master_addrs=127.23.44.254:44273
--webserver_port=42103
--webserver_interface=127.23.44.194
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:44:08.991036 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:08.990551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:58381)
0926 20:44:08.990767 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:08.990872 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260926 20:44:08.992195 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:08.991716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:50585)
0926 20:44:08.991904 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:08.992005 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 20:44:08.992470 8107 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 20:44:09.339342 8111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:09.339673 8111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:09.339839 8111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:09.348102 8111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:09.348246 8111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:09.348450 8111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:44:09.359926 8111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:35231
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=42103
--tserver_master_addrs=127.23.44.254:44273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:09.361439 8111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:09.362847 8111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:09.378976 8116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:09.379055 8117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:09.379058 8119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:09.380241 8111 server_base.cc:1034] running on GCE node
I20260926 20:44:09.381044 8111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:09.382155 8111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:09.383416 8111 hybrid_clock.cc:648] HybridClock initialized: now 1790455449383379 us; error 32 us; skew 500 ppm
I20260926 20:44:09.383793 8111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:09 dist-test-slave-4d5t krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455449, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:09.387208 8111 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:44:09.389058 8111 webserver.cc:460] Webserver started at http://127.23.44.194:42103/ using document root <none> and password file <none>
I20260926 20:44:09.389808 8111 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:09.389923 8111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:09.396030 8111 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260926 20:44:09.399494 8126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:09.401257 8111 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260926 20:44:09.401464 8111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "225cc05acc474d20927e9b25885d47f1"
format_stamp: "Formatted at 2026-09-26 20:43:22 on dist-test-slave-4d5t"
server_key: "71adafdb2034e3995aff77dfaca994f7"
server_key_iv: "d66c35bae1adac2cff1a7bea8f73d73e"
server_key_version: "encryptionkey@0"
I20260926 20:44:09.402506 8111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:09.422416 8111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:09.429005 8111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:09.429523 8111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:09.430984 8111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:09.433034 8111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:09.433209 8111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:09.433363 8111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:09.433434 8111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:09.471444 8111 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:35231
I20260926 20:44:09.471566 8238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:35231 every 8 connection(s)
I20260926 20:44:09.473090 8111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:44:09.478175 8241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:09.476170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:56125)
0926 20:44:09.477052 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:09.477300 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":372,"threads_started":1}
W20260926 20:44:09.478958 8239 heartbeater.cc:644] Failed to heartbeat to 127.23.44.254:44273 (0 consecutive failures): Network error: Failed to ping master at 127.23.44.254:44273: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
I20260926 20:44:09.479913 8241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:09.479564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:43557)
0926 20:44:09.479731 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:09.479799 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 20:44:09.479938 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 8111
I20260926 20:44:09.480701 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:41499
--local_ip_for_outbound_sockets=127.23.44.195
--tserver_master_addrs=127.23.44.254:44273
--webserver_port=42027
--webserver_interface=127.23.44.195
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:44:09.481067 8241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:09.480636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:53201)
0926 20:44:09.480804 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:09.480930 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 20:44:09.481349 8239 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 20:44:09.832055 8242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:09.832376 8242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:09.832531 8242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:09.840065 8242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:09.840195 8242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:09.840387 8242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:44:09.851327 8242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:41499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=42027
--tserver_master_addrs=127.23.44.254:44273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:09.852762 8242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:09.854288 8242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:09.867835 8247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:09.867880 8248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:09.867934 8250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:09.868861 8242 server_base.cc:1034] running on GCE node
I20260926 20:44:09.869498 8242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:09.870827 8242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:09.872242 8242 hybrid_clock.cc:648] HybridClock initialized: now 1790455449872163 us; error 64 us; skew 500 ppm
I20260926 20:44:09.872694 8242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:09 dist-test-slave-4d5t krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455449, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:09.882061 8242 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:44:09.883723 8242 webserver.cc:460] Webserver started at http://127.23.44.195:42027/ using document root <none> and password file <none>
I20260926 20:44:09.884451 8242 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:09.884641 8242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:09.890175 8242 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:44:09.893452 8257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:09.895264 8242 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260926 20:44:09.895481 8242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "2e01cf8bdc2f4717b7a156574501b253"
format_stamp: "Formatted at 2026-09-26 20:43:23 on dist-test-slave-4d5t"
server_key: "339550196f560b5757b900dc042bd51c"
server_key_iv: "25000752c24ecf76486c33960b210fdc"
server_key_version: "encryptionkey@0"
I20260926 20:44:09.896538 8242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:09.918161 8242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:09.922848 8242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:09.923148 8242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:09.924801 8242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:09.926918 8242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:09.927102 8242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:09.927258 8242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:09.927350 8242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:09.962759 8242 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:41499
I20260926 20:44:09.962842 8369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:41499 every 8 connection(s)
I20260926 20:44:09.964246 8242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:44:09.968807 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 8242
I20260926 20:44:09.969945 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:09.967431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:51347)
0926 20:44:09.968587 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:09.968914 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":367,"threads_started":1}
W20260926 20:44:09.970656 8370 heartbeater.cc:644] Failed to heartbeat to 127.23.44.254:44273 (0 consecutive failures): Network error: Failed to ping master at 127.23.44.254:44273: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
I20260926 20:44:09.971465 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:09.971141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:41495)
0926 20:44:09.971278 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:09.971347 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 20:44:09.972743 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:09.972321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:52489)
0926 20:44:09.972476 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:09.972618 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 20:44:09.973136 8370 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 20:44:09.994200 8374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:09.993129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:47001)
0926 20:44:09.993877 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:09.993971 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":420,"threads_started":1}
I20260926 20:44:10.482710 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:10.481920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:46949)
0926 20:44:10.482378 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:10.482479 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":220,"threads_started":1}
I20260926 20:44:10.517036 8376 raft_consensus.cc:482] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:44:10.517388 8376 raft_consensus.cc:504] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:44:10.518944 8376 leader_election.cc:288] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231), 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499)
Sep 26 20:44:10 dist-test-slave-4d5t krb5kdc[6786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455448, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
Sep 26 20:44:10 dist-test-slave-4d5t krb5kdc[6786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455448, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:44:10.561164 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:10.519562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:60121 (local address 127.23.44.195:41499)
0926 20:44:10.533725 (+ 14163us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:10.533747 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:10.534208 (+ 461us) server_negotiation.cc:407] Connection header received
0926 20:44:10.536629 (+ 2421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:10.536643 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:10.537583 (+ 940us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:10.538041 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:10.541895 (+ 3854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:10.543867 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:10.547122 (+ 3255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:10.547861 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:10.548084 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:10.552666 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:10.552676 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:10.552695 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:10.552748 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:10.556667 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:10.557971 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:10.557977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:10.557982 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:10.558218 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:10.558676 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:10.558680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:10.558684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:10.559189 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:10.559401 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:10.560234 (+ 833us) server_negotiation.cc:299] Negotiation successful
0926 20:44:10.560719 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13914,"thread_start_us":13693,"threads_started":1}
I20260926 20:44:10.561228 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:10.519326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:41499 (local address 127.23.44.193:60121)
0926 20:44:10.534051 (+ 14725us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:10.534072 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:10.534195 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:10.538461 (+ 4266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:10.538474 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:10.539843 (+ 1369us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:10.541616 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:10.541649 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:10.544154 (+ 2505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:10.544159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:10.546889 (+ 2730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:10.546909 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:10.548110 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:10.548118 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:10.548363 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:10.549311 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:44:10.549330 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:10.552153 (+ 2823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:10.557043 (+ 4890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:10.557049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:10.557053 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:10.557811 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:10.558375 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:10.558380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:10.558385 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:10.558553 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:10.559337 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:10.559344 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:10.559722 (+ 378us) client_negotiation.cc:769] Sending connection context
0926 20:44:10.560001 (+ 279us) client_negotiation.cc:240] Negotiation successful
0926 20:44:10.560710 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14583,"mutex_wait_us":583,"spinlock_wait_cycles":32384,"thread_start_us":13348,"threads_started":1}
I20260926 20:44:10.561759 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:10.519713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:44603 (local address 127.23.44.194:35231)
0926 20:44:10.532184 (+ 12471us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:10.532212 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:10.533370 (+ 1158us) server_negotiation.cc:407] Connection header received
0926 20:44:10.536678 (+ 3308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:10.536687 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:10.537583 (+ 896us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:10.538041 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:10.541895 (+ 3854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:10.543867 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:10.547076 (+ 3209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:10.547815 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:10.548002 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:10.552818 (+ 4816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:10.552829 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:10.552846 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:10.552900 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:10.556309 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:10.557493 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:10.557503 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:10.557511 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:10.557737 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:10.558292 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:10.558298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:10.558301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:10.559252 (+ 951us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:10.559570 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:10.560365 (+ 795us) server_negotiation.cc:299] Negotiation successful
0926 20:44:10.560951 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12160,"thread_start_us":12517,"threads_started":1}
I20260926 20:44:10.562404 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:10.519325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:35231 (local address 127.23.44.193:44603)
0926 20:44:10.533119 (+ 13794us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:10.533181 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:10.534057 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:10.538461 (+ 4404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:10.538474 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:10.539847 (+ 1373us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:10.541616 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:10.541649 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:10.544149 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:10.544156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:10.546889 (+ 2733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:10.546909 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:10.548045 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:10.548053 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:10.548369 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:10.549264 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:44:10.549296 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:10.552295 (+ 2999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:10.556514 (+ 4219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:10.556526 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:10.556545 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:10.557214 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:10.557880 (+ 666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:10.557887 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:10.557890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:10.558156 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:10.559481 (+ 1325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:10.559487 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:10.559726 (+ 239us) client_negotiation.cc:769] Sending connection context
0926 20:44:10.560003 (+ 277us) client_negotiation.cc:240] Negotiation successful
0926 20:44:10.560958 (+ 955us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13589,"mutex_wait_us":661,"spinlock_wait_cycles":13056,"thread_start_us":14372,"threads_started":1}
W20260926 20:44:10.564797 7995 leader_election.cc:341] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499): Not found: Tablet not found: a92ad71b81a64552be31c4c4f66f9363
W20260926 20:44:10.565096 7996 leader_election.cc:341] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231): Not found: Tablet not found: a92ad71b81a64552be31c4c4f66f9363
I20260926 20:44:10.565294 7996 leader_election.cc:302] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6412b3674fcc4595bd37f96961c9dd3a; no voters: 225cc05acc474d20927e9b25885d47f1, 2e01cf8bdc2f4717b7a156574501b253
I20260926 20:44:10.565747 8376 raft_consensus.cc:2726] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 20:44:10.974676 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:10.973747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:49573)
0926 20:44:10.974283 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:10.974432 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":296,"threads_started":1}
I20260926 20:44:10.995654 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:10.995173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:57979)
0926 20:44:10.995389 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:10.995493 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260926 20:44:11.484992 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:11.483922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:44467)
0926 20:44:11.484645 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:11.484790 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":334,"threads_started":1}
I20260926 20:44:11.976774 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:11.975936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:41131)
0926 20:44:11.976442 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:11.976531 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":273,"threads_started":1}
I20260926 20:44:11.997582 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:11.996777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:42539)
0926 20:44:11.997288 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:11.997383 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":256,"threads_started":1}
I20260926 20:44:12.486734 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:12.485967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:48599)
0926 20:44:12.486461 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:12.486549 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":281,"threads_started":1}
I20260926 20:44:12.559393 8386 raft_consensus.cc:482] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:44:12.559594 8386 raft_consensus.cc:504] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:44:12.560295 8386 leader_election.cc:288] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231), 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499)
W20260926 20:44:12.561578 7995 leader_election.cc:341] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499): Not found: Tablet not found: a92ad71b81a64552be31c4c4f66f9363
W20260926 20:44:12.561790 7996 leader_election.cc:341] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231): Not found: Tablet not found: a92ad71b81a64552be31c4c4f66f9363
I20260926 20:44:12.561936 7996 leader_election.cc:302] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6412b3674fcc4595bd37f96961c9dd3a; no voters: 225cc05acc474d20927e9b25885d47f1, 2e01cf8bdc2f4717b7a156574501b253
I20260926 20:44:12.562256 8386 raft_consensus.cc:2726] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 20:44:12.978925 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:12.978014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:48189)
0926 20:44:12.978626 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:12.978742 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":319,"threads_started":1}
I20260926 20:44:12.999699 8388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:12.998680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:42909)
0926 20:44:12.999263 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:12.999409 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":277,"threads_started":1}
I20260926 20:44:13.488932 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:13.487983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:50805)
0926 20:44:13.488532 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:13.488722 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":293,"threads_started":1}
I20260926 20:44:13.981356 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:13.980243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:47145)
0926 20:44:13.981010 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:13.981118 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":452,"threads_started":1}
I20260926 20:44:14.002686 8391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:14.001434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:58367)
0926 20:44:14.002272 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:14.002385 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":520,"threads_started":1}
I20260926 20:44:14.490677 8392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:14.489954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:36919)
0926 20:44:14.490428 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:14.490511 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":264,"threads_started":1}
I20260926 20:44:14.983310 8393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:14.982457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:55947)
0926 20:44:14.982985 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:14.983095 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":265,"threads_started":1}
I20260926 20:44:15.004793 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:15.003827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:45121)
0926 20:44:15.004430 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:15.004531 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":351,"threads_started":1}
I20260926 20:44:15.492980 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:15.491943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:42455)
0926 20:44:15.492678 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:15.492774 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":367,"threads_started":1}
I20260926 20:44:15.561404 8396 raft_consensus.cc:482] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:44:15.561729 8396 raft_consensus.cc:504] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:44:15.562841 8396 leader_election.cc:288] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231), 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499)
W20260926 20:44:15.563995 7995 leader_election.cc:341] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499): Not found: Tablet not found: a92ad71b81a64552be31c4c4f66f9363
W20260926 20:44:15.564213 7996 leader_election.cc:341] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231): Not found: Tablet not found: a92ad71b81a64552be31c4c4f66f9363
I20260926 20:44:15.564373 7996 leader_election.cc:302] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6412b3674fcc4595bd37f96961c9dd3a; no voters: 225cc05acc474d20927e9b25885d47f1, 2e01cf8bdc2f4717b7a156574501b253
I20260926 20:44:15.564764 8396 raft_consensus.cc:2726] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 20:44:15.985302 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:15.984360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:48341)
0926 20:44:15.984947 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:15.985065 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":310,"threads_started":1}
I20260926 20:44:16.007146 8398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:16.005757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:54015)
0926 20:44:16.006462 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:16.006594 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":415,"threads_started":1}
I20260926 20:44:16.494940 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:16.494103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:46307)
0926 20:44:16.494628 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:16.494729 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":227,"threads_started":1}
I20260926 20:44:16.987418 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:16.986624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:43831)
0926 20:44:16.987148 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:16.987232 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":256,"threads_started":1}
I20260926 20:44:17.009069 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:17.008208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:37639)
0926 20:44:17.008773 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:17.008874 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":247,"threads_started":1}
I20260926 20:44:17.497020 8402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:17.496119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:41665)
0926 20:44:17.496676 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:17.496788 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":299,"threads_started":1}
I20260926 20:44:17.989501 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:17.988662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:57021)
0926 20:44:17.989193 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:17.989287 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":299,"threads_started":1}
I20260926 20:44:18.010891 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:18.010099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:48873)
0926 20:44:18.010590 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:18.010690 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":251,"threads_started":1}
I20260926 20:44:18.499086 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:18.497964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:46883)
0926 20:44:18.498669 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:18.498801 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":324,"threads_started":1}
I20260926 20:44:18.991392 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:18.990547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:53565)
0926 20:44:18.991135 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:18.991236 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":252,"threads_started":1}
I20260926 20:44:19.012918 8407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:19.012037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:50823)
0926 20:44:19.012622 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:19.012729 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":345,"threads_started":1}
I20260926 20:44:19.501607 8408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:19.500672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:55195)
0926 20:44:19.501290 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:19.501404 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":344,"threads_started":1}
I20260926 20:44:19.513849 8409 raft_consensus.cc:482] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:44:19.514127 8409 raft_consensus.cc:504] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:44:19.514873 8409 leader_election.cc:288] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231), 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499)
W20260926 20:44:19.515873 7995 leader_election.cc:341] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499): Not found: Tablet not found: a92ad71b81a64552be31c4c4f66f9363
W20260926 20:44:19.516096 7996 leader_election.cc:341] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231): Not found: Tablet not found: a92ad71b81a64552be31c4c4f66f9363
I20260926 20:44:19.516225 7996 leader_election.cc:302] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6412b3674fcc4595bd37f96961c9dd3a; no voters: 225cc05acc474d20927e9b25885d47f1, 2e01cf8bdc2f4717b7a156574501b253
I20260926 20:44:19.516445 8409 raft_consensus.cc:2726] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 20:44:19.993252 8410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:19.992392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:52751)
0926 20:44:19.992976 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:19.993065 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":324,"threads_started":1}
I20260926 20:44:20.015230 8411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:20.014088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:51767)
0926 20:44:20.014655 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:20.014779 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":276,"threads_started":1}
I20260926 20:44:20.503486 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:20.502623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:58937)
0926 20:44:20.503130 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:20.503244 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":262,"threads_started":1}
I20260926 20:44:20.994894 8413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:20.994204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:39985)
0926 20:44:20.994640 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:20.994731 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":211,"threads_started":1}
I20260926 20:44:21.017246 8414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:21.016423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:51445)
0926 20:44:21.016987 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:21.017075 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":348,"threads_started":1}
I20260926 20:44:21.334388 8415 raft_consensus.cc:482] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:44:21.334646 8415 raft_consensus.cc:504] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225cc05acc474d20927e9b25885d47f1" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35231 } } peers { permanent_uuid: "2e01cf8bdc2f4717b7a156574501b253" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41499 } } peers { permanent_uuid: "6412b3674fcc4595bd37f96961c9dd3a" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46479 } }
I20260926 20:44:21.335438 8415 leader_election.cc:288] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231), 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499)
W20260926 20:44:21.337000 7996 leader_election.cc:341] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 225cc05acc474d20927e9b25885d47f1 (127.23.44.194:35231): Not found: Tablet not found: a92ad71b81a64552be31c4c4f66f9363
W20260926 20:44:21.337293 7995 leader_election.cc:341] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e01cf8bdc2f4717b7a156574501b253 (127.23.44.195:41499): Not found: Tablet not found: a92ad71b81a64552be31c4c4f66f9363
I20260926 20:44:21.337406 7995 leader_election.cc:302] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6412b3674fcc4595bd37f96961c9dd3a; no voters: 225cc05acc474d20927e9b25885d47f1, 2e01cf8bdc2f4717b7a156574501b253
I20260926 20:44:21.337702 8415 raft_consensus.cc:2726] T a92ad71b81a64552be31c4c4f66f9363 P 6412b3674fcc4595bd37f96961c9dd3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 20:44:21.505513 8416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:21.504719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:34683)
0926 20:44:21.505221 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:21.505315 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":272,"threads_started":1}
I20260926 20:44:21.996892 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:21.995995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:44349)
0926 20:44:21.996606 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:21.996715 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":329,"threads_started":1}
I20260926 20:44:22.019325 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:22.018296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:53971)
0926 20:44:22.018943 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:22.019073 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":328,"threads_started":1}
I20260926 20:44:22.507510 8419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:22.506581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:56797)
0926 20:44:22.507227 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:22.507333 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":250,"threads_started":1}
I20260926 20:44:22.999171 8420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:22.998225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:48967)
0926 20:44:22.998787 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:22.998881 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":313,"threads_started":1}
I20260926 20:44:23.021998 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:23.020835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:50001)
0926 20:44:23.021511 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:23.021653 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":321,"threads_started":1}
I20260926 20:44:23.509281 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:23.508453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:50383)
0926 20:44:23.508987 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:23.509074 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":292,"threads_started":1}
I20260926 20:44:24.001525 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:24.000515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:35373)
0926 20:44:24.001192 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:24.001308 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":282,"threads_started":1}
I20260926 20:44:24.024165 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:24.023335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.193:35219)
0926 20:44:24.023862 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:24.023956 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":236,"threads_started":1}
I20260926 20:44:24.511261 8425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:24.510474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.194:46449)
0926 20:44:24.510986 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:24.511092 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":291,"threads_started":1}
I20260926 20:44:24.971278 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 7937
I20260926 20:44:24.990197 8426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:24.972521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33762 (local address 127.23.44.193:46479)
0926 20:44:24.973737 (+ 1216us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:24.973745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:24.973796 (+ 51us) server_negotiation.cc:407] Connection header received
0926 20:44:24.973890 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:24.973900 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:24.974205 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:24.974403 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:24.976205 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:24.977360 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:24.978826 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:24.979545 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:24.979696 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:24.982752 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:24.982764 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:24.982781 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:24.982826 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:24.986738 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:24.987593 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:24.987601 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:24.987603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:24.987843 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:24.988294 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:24.988303 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:24.988307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:24.988808 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:24.989161 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:24.989516 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 20:44:24.989838 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1022,"thread_start_us":841,"threads_started":1}
I20260926 20:44:24.991513 8102 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:25.003682 8428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:25.002855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44273 (local address 127.23.44.195:47635)
0926 20:44:25.003371 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:25.003525 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.254:44273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":283,"threads_started":1}
I20260926 20:44:25.086253 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 7937
I20260926 20:44:25.106479 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 8111
I20260926 20:44:25.121804 8430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:25.107261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53114 (local address 127.23.44.194:35231)
0926 20:44:25.107758 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:25.107766 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:25.107785 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:44:25.107860 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:25.107863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:25.107941 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:25.108062 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:25.109512 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:25.110320 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:25.111559 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:25.111984 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:25.112127 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:25.115090 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:25.115096 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:25.115099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:25.115136 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:25.118458 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:25.119313 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:25.119320 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:25.119324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:25.119479 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:25.120048 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:25.120060 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:25.120063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:25.120431 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:25.120946 (+ 515us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:25.121084 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 20:44:25.121363 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":242,"threads_started":1}
I20260926 20:44:25.122637 8234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:25.212854 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 8111
I20260926 20:44:25.238322 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 8242
I20260926 20:44:25.254914 8432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:25.239491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54510 (local address 127.23.44.195:41499)
0926 20:44:25.240199 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:25.240204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:25.240226 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:44:25.240300 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:25.240305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:25.240389 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:25.240539 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:25.242145 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:25.243006 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:25.244373 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:25.244954 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:25.245072 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:25.247884 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:25.247889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:25.247893 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:25.247934 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:25.250838 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:25.252109 (+ 1271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:25.252116 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:25.252118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:25.252414 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:25.253375 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:25.253384 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:25.253387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:25.253761 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:25.254008 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:25.254376 (+ 368us) server_negotiation.cc:299] Negotiation successful
0926 20:44:25.254587 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":402,"threads_started":1}
I20260926 20:44:25.255733 8365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:25.343714 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 8242
2026-09-26T20:44:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65440 ms)
[----------] 30 tests from SecurityITest (313686 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:44:25 dist-test-slave-4d5t krb5kdc[8436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:44:25 dist-test-slave-4d5t krb5kdc[8436](info): set up 2 sockets
Sep 26 20:44:25 dist-test-slave-4d5t krb5kdc[8436](info): commencing operation
krb5kdc: starting...
W20260926 20:44:26.505545 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:44:26 dist-test-slave-4d5t krb5kdc[8436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:44:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:44:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:44:26.784428 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:44517
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:44517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:27.154333 8449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:27.154778 8449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:27.154973 8449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:27.163784 8449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:44:27.163954 8449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:27.164152 8449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:27.164297 8449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:44:27.164384 8449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:44:27.177081 8449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:44517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:44517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:27.178673 8449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:27.180161 8449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:27.193849 8457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:27.194154 8455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:27.194134 8454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:27.198441 8449 server_base.cc:1034] running on GCE node
I20260926 20:44:27.199486 8449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:27.201373 8449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:27.202648 8449 hybrid_clock.cc:648] HybridClock initialized: now 1790455467202575 us; error 63 us; skew 500 ppm
I20260926 20:44:27.203078 8449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:27 dist-test-slave-4d5t krb5kdc[8436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455467, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:27.206561 8449 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:44:27.208395 8449 webserver.cc:460] Webserver started at http://127.23.44.254:39967/ using document root <none> and password file <none>
I20260926 20:44:27.209281 8449 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:27.209438 8449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:27.209764 8449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:27.213079 8449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "aaac31adfa1a4f55828f6a7f909a45f4"
format_stamp: "Formatted at 2026-09-26 20:44:27 on dist-test-slave-4d5t"
server_key: "1e6cc23d776727f5bfc88ce6e127095b"
server_key_iv: "228eb295fa3d46a85df2ba5d44232de3"
server_key_version: "encryptionkey@0"
I20260926 20:44:27.213897 8449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "aaac31adfa1a4f55828f6a7f909a45f4"
format_stamp: "Formatted at 2026-09-26 20:44:27 on dist-test-slave-4d5t"
server_key: "1e6cc23d776727f5bfc88ce6e127095b"
server_key_iv: "228eb295fa3d46a85df2ba5d44232de3"
server_key_version: "encryptionkey@0"
I20260926 20:44:27.219946 8449 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260926 20:44:27.223912 8464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:27.225801 8449 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:44:27.226030 8449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "aaac31adfa1a4f55828f6a7f909a45f4"
format_stamp: "Formatted at 2026-09-26 20:44:27 on dist-test-slave-4d5t"
server_key: "1e6cc23d776727f5bfc88ce6e127095b"
server_key_iv: "228eb295fa3d46a85df2ba5d44232de3"
server_key_version: "encryptionkey@0"
I20260926 20:44:27.226277 8449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:27.258028 8449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:27.264592 8449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:27.265033 8449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:27.286674 8449 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:44517
I20260926 20:44:27.286712 8515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:44517 every 8 connection(s)
I20260926 20:44:27.288125 8449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:44:27.292207 8516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:27.292483 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 8449
I20260926 20:44:27.292742 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:44:27.293567 23731 external_mini_cluster.cc:1428] Setting key 3446e8175d4d0ddf95e2a6cccb0d2371
I20260926 20:44:27.309005 8516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4: Bootstrap starting.
Sep 26 20:44:27 dist-test-slave-4d5t krb5kdc[8436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:44:27.313354 8516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:27.314766 8516 log.cc:824] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:27.318035 8516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4: No bootstrap required, opened a new log
I20260926 20:44:27.320300 8519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:27.296296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48498 (local address 127.23.44.254:44517)
0926 20:44:27.297461 (+ 1165us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:27.297476 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:27.297533 (+ 57us) server_negotiation.cc:407] Connection header received
0926 20:44:27.299725 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:27.299766 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:27.300302 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:27.300811 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:27.302876 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:27.304246 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:27.306469 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:27.307194 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:27.307339 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:27.312621 (+ 5282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:27.312631 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:27.312648 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:27.312696 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:27.316055 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:27.316909 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:27.316916 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:27.316922 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:27.317089 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:27.317508 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:27.317512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:27.317514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:27.318051 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:27.318374 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:27.318853 (+ 479us) server_negotiation.cc:299] Negotiation successful
0926 20:44:27.319291 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":570,"threads_started":1}
I20260926 20:44:27.327255 8516 raft_consensus.cc:348] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaac31adfa1a4f55828f6a7f909a45f4" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44517 } }
I20260926 20:44:27.327664 8516 raft_consensus.cc:374] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:27.327788 8516 raft_consensus.cc:729] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaac31adfa1a4f55828f6a7f909a45f4, State: Initialized, Role: FOLLOWER
I20260926 20:44:27.328435 8516 consensus_queue.cc:260] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaac31adfa1a4f55828f6a7f909a45f4" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44517 } }
I20260926 20:44:27.328794 8516 raft_consensus.cc:388] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:44:27.328922 8516 raft_consensus.cc:482] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:44:27.329061 8516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:27.331936 8516 raft_consensus.cc:504] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaac31adfa1a4f55828f6a7f909a45f4" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44517 } }
I20260926 20:44:27.332480 8516 leader_election.cc:302] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aaac31adfa1a4f55828f6a7f909a45f4; no voters:
I20260926 20:44:27.333294 8516 leader_election.cc:288] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:44:27.333673 8521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:27.334527 8521 raft_consensus.cc:686] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [term 1 LEADER]: Becoming Leader. State: Replica: aaac31adfa1a4f55828f6a7f909a45f4, State: Running, Role: LEADER
I20260926 20:44:27.335175 8521 consensus_queue.cc:237] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaac31adfa1a4f55828f6a7f909a45f4" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44517 } }
I20260926 20:44:27.335871 8516 sys_catalog.cc:564] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:44:27.338205 8523 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aaac31adfa1a4f55828f6a7f909a45f4. Latest consensus state: current_term: 1 leader_uuid: "aaac31adfa1a4f55828f6a7f909a45f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaac31adfa1a4f55828f6a7f909a45f4" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44517 } } }
I20260926 20:44:27.338224 8522 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aaac31adfa1a4f55828f6a7f909a45f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaac31adfa1a4f55828f6a7f909a45f4" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44517 } } }
I20260926 20:44:27.338791 8522 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:27.338969 8523 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4 [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:27.340534 8526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:44:27.347033 8526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:44:27.356381 8526 catalog_manager.cc:1347] Generated new cluster ID: c301e50f3d024b2e8dd6b2102918c167
I20260926 20:44:27.356516 8526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:44:27.384289 8526 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:44:27.386477 8526 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:44:27.395705 8526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4: Generated new TSK 0
I20260926 20:44:27.396863 8526 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:44:27.472651 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:44517
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:27.840760 8542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:27.841086 8542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:27.841252 8542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:27.850750 8542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:27.850888 8542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:27.851094 8542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:44:27.861440 8542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:44517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:27.862788 8542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:27.864094 8542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:27.877875 8550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:27.878067 8547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:27.878026 8548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:27.879066 8542 server_base.cc:1034] running on GCE node
I20260926 20:44:27.879786 8542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:27.880975 8542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:27.882244 8542 hybrid_clock.cc:648] HybridClock initialized: now 1790455467882195 us; error 55 us; skew 500 ppm
I20260926 20:44:27.882616 8542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:27 dist-test-slave-4d5t krb5kdc[8436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455467, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:27.885965 8542 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:44:27.887495 8542 webserver.cc:460] Webserver started at http://127.23.44.193:46457/ using document root <none> and password file <none>
I20260926 20:44:27.888168 8542 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:27.888276 8542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:27.888532 8542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:27.893047 8542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "1936406aa5ce43959990ac3acd203bf5"
format_stamp: "Formatted at 2026-09-26 20:44:27 on dist-test-slave-4d5t"
server_key: "1ddc5aad55ce16079de6d01422534613"
server_key_iv: "7ed0c677382156a036e3d56f559c8cda"
server_key_version: "encryptionkey@0"
I20260926 20:44:27.894297 8542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "1936406aa5ce43959990ac3acd203bf5"
format_stamp: "Formatted at 2026-09-26 20:44:27 on dist-test-slave-4d5t"
server_key: "1ddc5aad55ce16079de6d01422534613"
server_key_iv: "7ed0c677382156a036e3d56f559c8cda"
server_key_version: "encryptionkey@0"
I20260926 20:44:27.900736 8542 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260926 20:44:27.905404 8557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:27.907266 8542 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:44:27.907509 8542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "1936406aa5ce43959990ac3acd203bf5"
format_stamp: "Formatted at 2026-09-26 20:44:27 on dist-test-slave-4d5t"
server_key: "1ddc5aad55ce16079de6d01422534613"
server_key_iv: "7ed0c677382156a036e3d56f559c8cda"
server_key_version: "encryptionkey@0"
I20260926 20:44:27.907871 8542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:27.932428 8542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:27.937981 8542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:27.938298 8542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:27.939334 8542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:27.940963 8542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:27.941042 8542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:27.941170 8542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:27.941246 8542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:27.980584 8542 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:46857
I20260926 20:44:27.980630 8669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:46857 every 8 connection(s)
I20260926 20:44:27.981935 8542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:44:27.991986 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 8542
I20260926 20:44:27.992228 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:44:27.992918 23731 external_mini_cluster.cc:1428] Setting key 37f670877fe43c2db7ccfa3e08796c39
Sep 26 20:44:27 dist-test-slave-4d5t krb5kdc[8436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455467, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:44:28.005693 8672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:27.984737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:54925 (local address 127.23.44.254:44517)
0926 20:44:27.985353 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:27.985361 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:27.986221 (+ 860us) server_negotiation.cc:407] Connection header received
0926 20:44:27.988262 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:27.988267 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:27.988372 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:27.988489 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:27.991100 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:27.991899 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:27.993941 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:27.994366 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:27.994511 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:27.998824 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:27.998830 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:27.998834 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:27.998870 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:28.001534 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:28.002562 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:28.002567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:28.002569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:28.002718 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:28.003421 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:28.003429 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:28.003435 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:28.003832 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:28.004004 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:28.004925 (+ 921us) server_negotiation.cc:299] Negotiation successful
0926 20:44:28.005196 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":280,"threads_started":1}
I20260926 20:44:28.006332 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:27.985156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44517 (local address 127.23.44.193:54925)
0926 20:44:27.986050 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:27.986093 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:27.987946 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:27.988847 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:27.988856 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:27.989859 (+ 1003us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:27.990820 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:27.990853 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:27.992123 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:27.992128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:27.993776 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:27.993792 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:27.994631 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:27.994640 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:27.994904 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:27.995621 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:44:27.995649 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:27.998523 (+ 2874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:28.001702 (+ 3179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:28.001715 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:28.001737 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:28.002397 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:28.002842 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:28.002850 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:28.002856 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:28.003241 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:28.003998 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:28.004005 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:28.004354 (+ 349us) client_negotiation.cc:769] Sending connection context
0926 20:44:28.004774 (+ 420us) client_negotiation.cc:240] Negotiation successful
0926 20:44:28.005188 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":380,"threads_started":1}
I20260926 20:44:28.008095 8670 heartbeater.cc:347] Connected to a master server at 127.23.44.254:44517
I20260926 20:44:28.008424 8670 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:28.009469 8670 heartbeater.cc:511] Master 127.23.44.254:44517 requested a full tablet report, sending...
I20260926 20:44:28.012324 8481 ts_manager.cc:194] Registered new tserver with Master: 1936406aa5ce43959990ac3acd203bf5 (127.23.44.193:46857)
I20260926 20:44:28.013967 8481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:54925
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:44:28.056221 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:44517
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:28.407495 8676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:28.407837 8676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:28.408012 8676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:28.415786 8676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:28.415956 8676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:28.416131 8676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:44:28.426785 8676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:44517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:28.428177 8676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:28.429518 8676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:28.443547 8681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:28.443940 8682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:28.444164 8684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:28.445165 8676 server_base.cc:1034] running on GCE node
I20260926 20:44:28.445868 8676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:28.446919 8676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:28.448194 8676 hybrid_clock.cc:648] HybridClock initialized: now 1790455468448149 us; error 71 us; skew 500 ppm
I20260926 20:44:28.448621 8676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:28 dist-test-slave-4d5t krb5kdc[8436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455468, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:28.456144 8676 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:44:28.457904 8676 webserver.cc:460] Webserver started at http://127.23.44.194:38975/ using document root <none> and password file <none>
I20260926 20:44:28.458649 8676 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:28.458797 8676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:28.459133 8676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:28.462234 8676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb"
format_stamp: "Formatted at 2026-09-26 20:44:28 on dist-test-slave-4d5t"
server_key: "e8d93e064e4236c26d33b751739afd47"
server_key_iv: "80eb9ce22754b3cf41a7e71b75d3b73f"
server_key_version: "encryptionkey@0"
I20260926 20:44:28.463093 8676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb"
format_stamp: "Formatted at 2026-09-26 20:44:28 on dist-test-slave-4d5t"
server_key: "e8d93e064e4236c26d33b751739afd47"
server_key_iv: "80eb9ce22754b3cf41a7e71b75d3b73f"
server_key_version: "encryptionkey@0"
I20260926 20:44:28.469858 8676 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260926 20:44:28.474290 8691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:28.476040 8676 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:44:28.476281 8676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb"
format_stamp: "Formatted at 2026-09-26 20:44:28 on dist-test-slave-4d5t"
server_key: "e8d93e064e4236c26d33b751739afd47"
server_key_iv: "80eb9ce22754b3cf41a7e71b75d3b73f"
server_key_version: "encryptionkey@0"
I20260926 20:44:28.476521 8676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:28.501744 8676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:28.506615 8676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:28.506922 8676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:28.507992 8676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:28.509497 8676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:28.509574 8676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:28.509662 8676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:28.509709 8676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:28.546896 8676 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:45411
I20260926 20:44:28.547075 8803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:45411 every 8 connection(s)
I20260926 20:44:28.548373 8676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:44:28.555698 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 8676
I20260926 20:44:28.556106 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:44:28.556720 23731 external_mini_cluster.cc:1428] Setting key c2f3142c64681ce847199d7b59b0d76d
Sep 26 20:44:28 dist-test-slave-4d5t krb5kdc[8436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455468, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:44:28.570348 8806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:28.551482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:50441 (local address 127.23.44.254:44517)
0926 20:44:28.552230 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:28.552239 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:28.553183 (+ 944us) server_negotiation.cc:407] Connection header received
0926 20:44:28.555022 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:28.555028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:28.555119 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:28.555265 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:28.557116 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:28.557863 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:28.559462 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:28.559911 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:28.560060 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:28.564183 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:28.564189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:28.564193 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:28.564231 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:28.566996 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:28.567829 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:28.567834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:28.567838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:28.567973 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:28.568423 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:28.568432 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:28.568434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:28.568799 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:28.568960 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:28.569704 (+ 744us) server_negotiation.cc:299] Negotiation successful
0926 20:44:28.569954 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":425,"threads_started":1}
I20260926 20:44:28.571372 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:28.552118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44517 (local address 127.23.44.194:50441)
0926 20:44:28.553012 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:28.553056 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:28.554733 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:28.555519 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:28.555527 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:28.556162 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:28.556938 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:28.556957 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:28.558025 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:28.558035 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:28.559295 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:28.559307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:28.560152 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:28.560159 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:28.560410 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:28.561310 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:44:28.561336 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:28.563893 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:28.567182 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:28.567192 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:28.567223 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:28.567689 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:28.568090 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:28.568096 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:28.568100 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:28.568299 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:28.568938 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:28.568946 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:28.569328 (+ 382us) client_negotiation.cc:769] Sending connection context
0926 20:44:28.569605 (+ 277us) client_negotiation.cc:240] Negotiation successful
0926 20:44:28.570175 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":435,"threads_started":1}
I20260926 20:44:28.573462 8804 heartbeater.cc:347] Connected to a master server at 127.23.44.254:44517
I20260926 20:44:28.573868 8804 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:28.574849 8804 heartbeater.cc:511] Master 127.23.44.254:44517 requested a full tablet report, sending...
I20260926 20:44:28.577122 8481 ts_manager.cc:194] Registered new tserver with Master: b0fef3fe04be487b9f1301aaf9c5e7eb (127.23.44.194:45411)
I20260926 20:44:28.578483 8481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:50441
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:44:28.616304 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:44517
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:28.953543 8810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:28.953840 8810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:28.953999 8810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:28.961381 8810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:28.961510 8810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:28.961707 8810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:44:28.972395 8810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:44517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:28.974610 8810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:28.976516 8810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:28.992671 8815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:28.992725 8816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:28.992763 8818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:28.993916 8810 server_base.cc:1034] running on GCE node
I20260926 20:44:28.994638 8810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:28.995739 8810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:28.997035 8810 hybrid_clock.cc:648] HybridClock initialized: now 1790455468996974 us; error 72 us; skew 500 ppm
I20260926 20:44:28.997728 8810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:28 dist-test-slave-4d5t krb5kdc[8436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455468, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:29.001714 8810 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:44:29.003539 8810 webserver.cc:460] Webserver started at http://127.23.44.195:44839/ using document root <none> and password file <none>
I20260926 20:44:29.004347 8810 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:29.004513 8810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:29.004899 8810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:29.008138 8810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "021aa06c93c14f678fc3f4064776ca40"
format_stamp: "Formatted at 2026-09-26 20:44:29 on dist-test-slave-4d5t"
server_key: "c6392cdeebb87726bfdd472212781c38"
server_key_iv: "d4572f5f9fcd676dd5d48a14889b1960"
server_key_version: "encryptionkey@0"
I20260926 20:44:29.009028 8810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "021aa06c93c14f678fc3f4064776ca40"
format_stamp: "Formatted at 2026-09-26 20:44:29 on dist-test-slave-4d5t"
server_key: "c6392cdeebb87726bfdd472212781c38"
server_key_iv: "d4572f5f9fcd676dd5d48a14889b1960"
server_key_version: "encryptionkey@0"
I20260926 20:44:29.015214 8810 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.005s
I20260926 20:44:29.017804 8670 heartbeater.cc:503] Master 127.23.44.254:44517 was elected leader, sending a full tablet report...
I20260926 20:44:29.019865 8825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:29.021773 8810 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260926 20:44:29.021963 8810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "021aa06c93c14f678fc3f4064776ca40"
format_stamp: "Formatted at 2026-09-26 20:44:29 on dist-test-slave-4d5t"
server_key: "c6392cdeebb87726bfdd472212781c38"
server_key_iv: "d4572f5f9fcd676dd5d48a14889b1960"
server_key_version: "encryptionkey@0"
I20260926 20:44:29.022171 8810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:29.036732 8810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:29.041179 8810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:29.041504 8810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:29.042596 8810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:29.044153 8810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:29.044262 8810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:29.044417 8810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:29.044493 8810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:29.089452 8810 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:41871
I20260926 20:44:29.089494 8937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:41871 every 8 connection(s)
I20260926 20:44:29.090768 8810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:44:29.094602 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 8810
I20260926 20:44:29.094837 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:44:29.095466 23731 external_mini_cluster.cc:1428] Setting key ec1306f4c1925d0c95f76d0838523612
Sep 26 20:44:29 dist-test-slave-4d5t krb5kdc[8436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455468, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:44:29.113945 8940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.093503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:53193 (local address 127.23.44.254:44517)
0926 20:44:29.093963 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:29.093970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:29.095202 (+ 1232us) server_negotiation.cc:407] Connection header received
0926 20:44:29.097737 (+ 2535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:29.097743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:29.097864 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:29.097976 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:29.099985 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.101034 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.103038 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.103596 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.103734 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.107505 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:29.107511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:29.107515 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:29.107557 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:29.110612 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:29.111399 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:29.111406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:29.111410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:29.111539 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:29.111987 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:29.111993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:29.111997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:29.112290 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:29.112456 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:29.113284 (+ 828us) server_negotiation.cc:299] Negotiation successful
0926 20:44:29.113524 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":225,"threads_started":1}
I20260926 20:44:29.114877 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.094065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44517 (local address 127.23.44.195:53193)
0926 20:44:29.094997 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:29.095066 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:29.097350 (+ 2284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:29.098293 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:29.098302 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:29.098989 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:29.099725 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:29.099744 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.101325 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.101332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:29.102829 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:29.102849 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.103884 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.103889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:29.104102 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.104908 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:44:29.104933 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:29.107243 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:29.110778 (+ 3535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:29.110789 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:29.110808 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:29.111260 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:29.111653 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:29.111658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:29.111662 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:29.111869 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:29.112430 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:29.112439 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:29.112905 (+ 466us) client_negotiation.cc:769] Sending connection context
0926 20:44:29.113181 (+ 276us) client_negotiation.cc:240] Negotiation successful
0926 20:44:29.113749 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":330,"threads_started":1}
I20260926 20:44:29.116350 8938 heartbeater.cc:347] Connected to a master server at 127.23.44.254:44517
I20260926 20:44:29.116724 8938 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:29.117475 8938 heartbeater.cc:511] Master 127.23.44.254:44517 requested a full tablet report, sending...
I20260926 20:44:29.119099 8481 ts_manager.cc:194] Registered new tserver with Master: 021aa06c93c14f678fc3f4064776ca40 (127.23.44.195:41871)
I20260926 20:44:29.119930 8481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:53193
I20260926 20:44:29.124845 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:44:29.145578 8940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.129905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48512 (local address 127.23.44.254:44517)
0926 20:44:29.130208 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:29.130214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:29.130709 (+ 495us) server_negotiation.cc:407] Connection header received
0926 20:44:29.130833 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:29.130836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:29.130944 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:29.131055 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:29.132684 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.133460 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.135137 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.135678 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.135920 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.138804 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:29.138809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:29.138813 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:29.138858 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:29.142366 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:29.143373 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:29.143379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:29.143381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:29.143568 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:29.143971 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:29.143974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:29.143976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:29.144356 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:29.144644 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:29.145123 (+ 479us) server_negotiation.cc:299] Negotiation successful
0926 20:44:29.145339 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260926 20:44:29.150840 8481 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:44:29.159610 8481 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:44:29.187479 8952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.175837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58252 (local address 127.23.44.195:41871)
0926 20:44:29.177717 (+ 1880us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:29.177725 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:29.177758 (+ 33us) server_negotiation.cc:407] Connection header received
0926 20:44:29.177854 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:29.177863 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:29.178118 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:29.179154 (+ 1036us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:29.179448 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.181112 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.184044 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.185593 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.185727 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.185981 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:29.186873 (+ 892us) server_negotiation.cc:299] Negotiation successful
0926 20:44:29.187074 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1727,"thread_start_us":1588,"threads_started":1}
I20260926 20:44:29.188887 8951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.175627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:41871 (local address 127.0.0.1:58252)
0926 20:44:29.176487 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:29.176529 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:29.176746 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:29.178393 (+ 1647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:29.178407 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:29.178443 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:29.178948 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:29.178967 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.181260 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.181266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:29.183887 (+ 2621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:29.183905 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.186144 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.186152 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:29.186510 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.186580 (+ 70us) client_negotiation.cc:769] Sending connection context
0926 20:44:29.188117 (+ 1537us) client_negotiation.cc:240] Negotiation successful
0926 20:44:29.188379 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":56,"thread_start_us":353,"threads_started":1}
I20260926 20:44:29.189230 8954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.176447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42766 (local address 127.23.44.193:46857)
0926 20:44:29.177474 (+ 1027us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:29.177487 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:29.178920 (+ 1433us) server_negotiation.cc:407] Connection header received
0926 20:44:29.179025 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:29.179035 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:29.179267 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:29.179448 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:29.180358 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.182004 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.184818 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.187266 (+ 2448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.187408 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.187639 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:29.188716 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0926 20:44:29.188916 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":476,"threads_started":1}
I20260926 20:44:29.190009 8953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.176222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:46857 (local address 127.0.0.1:42766)
0926 20:44:29.178620 (+ 2398us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:29.178643 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:29.178769 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:29.179693 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:29.179698 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:29.179724 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:29.180176 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:29.180188 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.182181 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.182186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:29.184623 (+ 2437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:29.184637 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.187813 (+ 3176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.187820 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:29.187999 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.188030 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 20:44:29.188125 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 20:44:29.189568 (+ 1443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2256,"mutex_wait_us":1,"thread_start_us":2118,"threads_started":1}
I20260926 20:44:29.193495 8605 tablet_service.cc:1467] Processing CreateTablet for tablet 47f71b481b5b4879ad502233c9b2c722 (DEFAULT_TABLE table=test-table [id=8bd482a7499d41209d99cabeb72a509d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:29.193495 8873 tablet_service.cc:1467] Processing CreateTablet for tablet 47f71b481b5b4879ad502233c9b2c722 (DEFAULT_TABLE table=test-table [id=8bd482a7499d41209d99cabeb72a509d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:29.195068 8873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47f71b481b5b4879ad502233c9b2c722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:29.195072 8605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47f71b481b5b4879ad502233c9b2c722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:29.196655 8955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.178425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:45411 (local address 127.0.0.1:58542)
0926 20:44:29.189084 (+ 10659us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:29.189104 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:29.189257 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:29.190443 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:29.190448 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:29.190480 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:29.190805 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:29.190817 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.192509 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.192516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:29.194345 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:29.194361 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.195778 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.195784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:29.195950 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.195989 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 20:44:29.196114 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 20:44:29.196249 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10532,"thread_start_us":10336,"threads_started":1}
I20260926 20:44:29.196647 8956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.178914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58542 (local address 127.23.44.194:45411)
0926 20:44:29.188584 (+ 9670us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:29.188594 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:29.189388 (+ 794us) server_negotiation.cc:407] Connection header received
0926 20:44:29.189503 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:29.189512 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:29.189831 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:29.190020 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:29.191005 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.192359 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.194529 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.195565 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.195729 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.195947 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:29.196154 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 20:44:29.196320 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9470,"thread_start_us":9340,"threads_started":1}
I20260926 20:44:29.200307 8739 tablet_service.cc:1467] Processing CreateTablet for tablet 47f71b481b5b4879ad502233c9b2c722 (DEFAULT_TABLE table=test-table [id=8bd482a7499d41209d99cabeb72a509d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:29.201622 8739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47f71b481b5b4879ad502233c9b2c722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:29.210626 8958 tablet_bootstrap.cc:492] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5: Bootstrap starting.
I20260926 20:44:29.211465 8957 tablet_bootstrap.cc:492] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40: Bootstrap starting.
I20260926 20:44:29.214381 8959 tablet_bootstrap.cc:492] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb: Bootstrap starting.
I20260926 20:44:29.215364 8957 tablet_bootstrap.cc:654] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:29.215397 8958 tablet_bootstrap.cc:654] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:29.216751 8958 log.cc:824] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:29.216723 8957 log.cc:824] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:29.218155 8959 tablet_bootstrap.cc:654] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:29.219518 8959 log.cc:824] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:29.219956 8957 tablet_bootstrap.cc:492] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40: No bootstrap required, opened a new log
I20260926 20:44:29.220235 8958 tablet_bootstrap.cc:492] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5: No bootstrap required, opened a new log
I20260926 20:44:29.220281 8957 ts_tablet_manager.cc:1397] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 20:44:29.220623 8958 ts_tablet_manager.cc:1397] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 20:44:29.222898 8959 tablet_bootstrap.cc:492] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb: No bootstrap required, opened a new log
I20260926 20:44:29.223151 8959 ts_tablet_manager.cc:1397] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 20:44:29.230722 8957 raft_consensus.cc:348] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021aa06c93c14f678fc3f4064776ca40" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41871 } } peers { permanent_uuid: "1936406aa5ce43959990ac3acd203bf5" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46857 } } peers { permanent_uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45411 } }
I20260926 20:44:29.231388 8957 raft_consensus.cc:374] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:29.231535 8957 raft_consensus.cc:729] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021aa06c93c14f678fc3f4064776ca40, State: Initialized, Role: FOLLOWER
I20260926 20:44:29.231594 8958 raft_consensus.cc:348] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021aa06c93c14f678fc3f4064776ca40" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41871 } } peers { permanent_uuid: "1936406aa5ce43959990ac3acd203bf5" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46857 } } peers { permanent_uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45411 } }
I20260926 20:44:29.232297 8958 raft_consensus.cc:374] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:29.232416 8958 raft_consensus.cc:729] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1936406aa5ce43959990ac3acd203bf5, State: Initialized, Role: FOLLOWER
I20260926 20:44:29.232407 8957 consensus_queue.cc:260] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021aa06c93c14f678fc3f4064776ca40" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41871 } } peers { permanent_uuid: "1936406aa5ce43959990ac3acd203bf5" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46857 } } peers { permanent_uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45411 } }
I20260926 20:44:29.233386 8958 consensus_queue.cc:260] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021aa06c93c14f678fc3f4064776ca40" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41871 } } peers { permanent_uuid: "1936406aa5ce43959990ac3acd203bf5" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46857 } } peers { permanent_uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45411 } }
I20260926 20:44:29.234203 8938 heartbeater.cc:503] Master 127.23.44.254:44517 was elected leader, sending a full tablet report...
I20260926 20:44:29.234964 8957 ts_tablet_manager.cc:1428] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260926 20:44:29.235497 8958 ts_tablet_manager.cc:1428] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260926 20:44:29.235870 8959 raft_consensus.cc:348] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021aa06c93c14f678fc3f4064776ca40" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41871 } } peers { permanent_uuid: "1936406aa5ce43959990ac3acd203bf5" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46857 } } peers { permanent_uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45411 } }
I20260926 20:44:29.236380 8959 raft_consensus.cc:374] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:29.236505 8959 raft_consensus.cc:729] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fef3fe04be487b9f1301aaf9c5e7eb, State: Initialized, Role: FOLLOWER
I20260926 20:44:29.237246 8959 consensus_queue.cc:260] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021aa06c93c14f678fc3f4064776ca40" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41871 } } peers { permanent_uuid: "1936406aa5ce43959990ac3acd203bf5" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46857 } } peers { permanent_uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45411 } }
I20260926 20:44:29.239624 8804 heartbeater.cc:503] Master 127.23.44.254:44517 was elected leader, sending a full tablet report...
I20260926 20:44:29.240464 8959 ts_tablet_manager.cc:1428] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
W20260926 20:44:29.300807 8805 tablet.cc:2369] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:44:29.326985 8964 raft_consensus.cc:482] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:44:29.327355 8964 raft_consensus.cc:504] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021aa06c93c14f678fc3f4064776ca40" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41871 } } peers { permanent_uuid: "1936406aa5ce43959990ac3acd203bf5" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46857 } } peers { permanent_uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45411 } }
I20260926 20:44:29.329257 8964 leader_election.cc:288] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 021aa06c93c14f678fc3f4064776ca40 (127.23.44.195:41871), b0fef3fe04be487b9f1301aaf9c5e7eb (127.23.44.194:45411)
I20260926 20:44:29.337493 8952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.329822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:41795 (local address 127.23.44.195:41871)
0926 20:44:29.330077 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:29.330086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:29.331068 (+ 982us) server_negotiation.cc:407] Connection header received
0926 20:44:29.331257 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:29.331260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:29.331318 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:29.331446 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:29.332122 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.332955 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.335407 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.336283 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.336422 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.336475 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:29.337074 (+ 599us) server_negotiation.cc:299] Negotiation successful
0926 20:44:29.337219 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 20:44:29.337774 8956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.329714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:44545 (local address 127.23.44.194:45411)
0926 20:44:29.329974 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:29.329983 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:29.330905 (+ 922us) server_negotiation.cc:407] Connection header received
0926 20:44:29.331196 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:29.331203 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:29.331293 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:29.331363 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:29.332201 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.333275 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.335462 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.336526 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.336750 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.336843 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:29.337314 (+ 471us) server_negotiation.cc:299] Negotiation successful
0926 20:44:29.337506 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:44:29.337942 8967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.329512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:45411 (local address 127.23.44.193:44545)
0926 20:44:29.330791 (+ 1279us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:29.330808 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:29.330998 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:29.331413 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:29.331418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:29.331515 (+ 97us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:29.332030 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:29.332058 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.333507 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.333511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:29.335238 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:29.335251 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.336904 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.336909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:29.337113 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.337153 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 20:44:29.337273 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 20:44:29.337429 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1174,"mutex_wait_us":8,"thread_start_us":912,"threads_started":1}
I20260926 20:44:29.338107 8966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.329561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:41871 (local address 127.23.44.193:41795)
0926 20:44:29.330965 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:29.330979 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:29.331126 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:29.331467 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:29.331472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:29.331541 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:29.331961 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:29.331974 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.333229 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.333238 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:29.335200 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:29.335221 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.336535 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.336541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:29.336851 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.336904 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 20:44:29.337055 (+ 151us) client_negotiation.cc:240] Negotiation successful
0926 20:44:29.337203 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1306,"mutex_wait_us":34,"spinlock_wait_cycles":57728,"thread_start_us":671,"threads_started":1}
I20260926 20:44:29.339221 8759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47f71b481b5b4879ad502233c9b2c722" candidate_uuid: "1936406aa5ce43959990ac3acd203bf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb" is_pre_election: true
I20260926 20:44:29.339560 8893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47f71b481b5b4879ad502233c9b2c722" candidate_uuid: "1936406aa5ce43959990ac3acd203bf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "021aa06c93c14f678fc3f4064776ca40" is_pre_election: true
I20260926 20:44:29.339910 8759 raft_consensus.cc:2454] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1936406aa5ce43959990ac3acd203bf5 in term 0.
I20260926 20:44:29.340070 8893 raft_consensus.cc:2454] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1936406aa5ce43959990ac3acd203bf5 in term 0.
I20260926 20:44:29.340767 8558 leader_election.cc:302] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 021aa06c93c14f678fc3f4064776ca40, 1936406aa5ce43959990ac3acd203bf5, b0fef3fe04be487b9f1301aaf9c5e7eb; no voters:
I20260926 20:44:29.341238 8964 raft_consensus.cc:2781] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:44:29.341408 8964 raft_consensus.cc:482] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:44:29.341531 8964 raft_consensus.cc:3037] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [term 0 FOLLOWER]: Advancing to term 1
W20260926 20:44:29.342525 8939 tablet.cc:2369] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:44:29.344264 8964 raft_consensus.cc:504] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021aa06c93c14f678fc3f4064776ca40" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41871 } } peers { permanent_uuid: "1936406aa5ce43959990ac3acd203bf5" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46857 } } peers { permanent_uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45411 } }
I20260926 20:44:29.345005 8964 leader_election.cc:288] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [CANDIDATE]: Term 1 election: Requested vote from peers 021aa06c93c14f678fc3f4064776ca40 (127.23.44.195:41871), b0fef3fe04be487b9f1301aaf9c5e7eb (127.23.44.194:45411)
I20260926 20:44:29.345502 8893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47f71b481b5b4879ad502233c9b2c722" candidate_uuid: "1936406aa5ce43959990ac3acd203bf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "021aa06c93c14f678fc3f4064776ca40"
I20260926 20:44:29.345680 8759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47f71b481b5b4879ad502233c9b2c722" candidate_uuid: "1936406aa5ce43959990ac3acd203bf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb"
I20260926 20:44:29.345755 8893 raft_consensus.cc:3037] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:29.345935 8759 raft_consensus.cc:3037] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:29.349517 8759 raft_consensus.cc:2454] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1936406aa5ce43959990ac3acd203bf5 in term 1.
I20260926 20:44:29.349517 8893 raft_consensus.cc:2454] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1936406aa5ce43959990ac3acd203bf5 in term 1.
I20260926 20:44:29.350064 8558 leader_election.cc:302] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 021aa06c93c14f678fc3f4064776ca40, 1936406aa5ce43959990ac3acd203bf5, b0fef3fe04be487b9f1301aaf9c5e7eb; no voters:
I20260926 20:44:29.350435 8964 raft_consensus.cc:2781] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:29.351027 8964 raft_consensus.cc:686] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [term 1 LEADER]: Becoming Leader. State: Replica: 1936406aa5ce43959990ac3acd203bf5, State: Running, Role: LEADER
I20260926 20:44:29.351629 8964 consensus_queue.cc:237] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021aa06c93c14f678fc3f4064776ca40" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41871 } } peers { permanent_uuid: "1936406aa5ce43959990ac3acd203bf5" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46857 } } peers { permanent_uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45411 } }
I20260926 20:44:29.357100 8481 catalog_manager.cc:5495] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1936406aa5ce43959990ac3acd203bf5 (127.23.44.193). New cstate: current_term: 1 leader_uuid: "1936406aa5ce43959990ac3acd203bf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021aa06c93c14f678fc3f4064776ca40" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1936406aa5ce43959990ac3acd203bf5" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 46857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45411 } health_report { overall_health: UNKNOWN } } }
I20260926 20:44:29.396680 8954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.389545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.23.44.193:46857)
0926 20:44:29.389769 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:29.389774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:29.389965 (+ 191us) server_negotiation.cc:407] Connection header received
0926 20:44:29.390131 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:29.390135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:29.390254 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:29.390458 (+ 204us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:44:29.390977 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.391756 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.393663 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.394117 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.394246 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.395136 (+ 890us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:44:29.395891 (+ 755us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:44:29.396144 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:29.396291 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 20:44:29.396381 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:44:29.406198 8893 raft_consensus.cc:1261] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40 [term 1 FOLLOWER]: Refusing update from remote peer 1936406aa5ce43959990ac3acd203bf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:44:29.406203 8759 raft_consensus.cc:1261] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb [term 1 FOLLOWER]: Refusing update from remote peer 1936406aa5ce43959990ac3acd203bf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:44:29.407588 8968 consensus_queue.cc:1035] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "021aa06c93c14f678fc3f4064776ca40" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:44:29.408026 8964 consensus_queue.cc:1035] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fef3fe04be487b9f1301aaf9c5e7eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 45411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:44:29.421504 8972 mvcc.cc:205] Tried to move back new op lower bound from 7333705602674626560 to 7333705602470322176. Current Snapshot: MvccSnapshot[applied={T|T < 7333705602674626560}]
I20260926 20:44:29.425611 8973 mvcc.cc:205] Tried to move back new op lower bound from 7333705602674626560 to 7333705602470322176. Current Snapshot: MvccSnapshot[applied={T|T < 7333705602674626560}]
I20260926 20:44:29.438376 8480 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:44:29.438723 8480 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:44:29.442971 8480 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aaac31adfa1a4f55828f6a7f909a45f4: Sending DeleteTablet for 3 replicas of tablet 47f71b481b5b4879ad502233c9b2c722
I20260926 20:44:29.444391 8873 tablet_service.cc:1514] Processing DeleteTablet for tablet 47f71b481b5b4879ad502233c9b2c722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:29 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:58252
I20260926 20:44:29.444396 8605 tablet_service.cc:1514] Processing DeleteTablet for tablet 47f71b481b5b4879ad502233c9b2c722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:29 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:42766
I20260926 20:44:29.445749 8739 tablet_service.cc:1514] Processing DeleteTablet for tablet 47f71b481b5b4879ad502233c9b2c722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:29 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:58542
I20260926 20:44:29.447566 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 8542
I20260926 20:44:29.459971 8980 tablet_replica.cc:330] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5: stopping tablet replica
I20260926 20:44:29.461091 8981 tablet_replica.cc:330] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40: stopping tablet replica
I20260926 20:44:29.461491 8981 raft_consensus.cc:2229] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:44:29.462049 8981 raft_consensus.cc:2258] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:29.460487 8980 raft_consensus.cc:2229] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:44:29.463068 8980 raft_consensus.cc:2258] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:29.465991 8980 ts_tablet_manager.cc:1905] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:29.469210 8981 ts_tablet_manager.cc:1905] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:29.473627 8980 ts_tablet_manager.cc:1918] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:29.473845 8980 log.cc:1196] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/47f71b481b5b4879ad502233c9b2c722
I20260926 20:44:29.474459 8980 ts_tablet_manager.cc:1939] T 47f71b481b5b4879ad502233c9b2c722 P 1936406aa5ce43959990ac3acd203bf5: Deleting consensus metadata
I20260926 20:44:29.474529 8982 tablet_replica.cc:330] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb: stopping tablet replica
I20260926 20:44:29.475459 8982 raft_consensus.cc:2229] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:44:29.476001 8982 raft_consensus.cc:2258] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:29.476573 8466 catalog_manager.cc:4841] TS 1936406aa5ce43959990ac3acd203bf5 (127.23.44.193:46857): tablet 47f71b481b5b4879ad502233c9b2c722 (table test-table [id=8bd482a7499d41209d99cabeb72a509d]) successfully deleted
I20260926 20:44:29.477026 8981 ts_tablet_manager.cc:1918] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:29.477233 8981 log.cc:1196] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/47f71b481b5b4879ad502233c9b2c722
I20260926 20:44:29.477866 8981 ts_tablet_manager.cc:1939] T 47f71b481b5b4879ad502233c9b2c722 P 021aa06c93c14f678fc3f4064776ca40: Deleting consensus metadata
I20260926 20:44:29.478999 8982 ts_tablet_manager.cc:1905] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:29.479873 8465 catalog_manager.cc:4841] TS 021aa06c93c14f678fc3f4064776ca40 (127.23.44.195:41871): tablet 47f71b481b5b4879ad502233c9b2c722 (table test-table [id=8bd482a7499d41209d99cabeb72a509d]) successfully deleted
Sep 26 20:44:29 dist-test-slave-4d5t krb5kdc[8436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:44:29.485860 8982 ts_tablet_manager.cc:1918] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:29.486057 8982 log.cc:1196] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/47f71b481b5b4879ad502233c9b2c722
I20260926 20:44:29.486712 8982 ts_tablet_manager.cc:1939] T 47f71b481b5b4879ad502233c9b2c722 P b0fef3fe04be487b9f1301aaf9c5e7eb: Deleting consensus metadata
I20260926 20:44:29.488708 8467 catalog_manager.cc:4841] TS b0fef3fe04be487b9f1301aaf9c5e7eb (127.23.44.194:45411): tablet 47f71b481b5b4879ad502233c9b2c722 (table test-table [id=8bd482a7499d41209d99cabeb72a509d]) successfully deleted
I20260926 20:44:29.493865 8954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.449408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42782 (local address 127.23.44.193:46857)
0926 20:44:29.449650 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:29.449659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:29.453374 (+ 3715us) server_negotiation.cc:407] Connection header received
0926 20:44:29.453473 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:29.453482 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:29.453611 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:29.453750 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:29.474065 (+ 20315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.475293 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.478021 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.478783 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.478898 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.485618 (+ 6720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:29.485634 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:29.485654 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:29.485713 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:29.490038 (+ 4325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:29.491238 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:29.491247 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:29.491251 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:29.491440 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:29.491984 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:29.491989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:29.491993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:29.492461 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:29.492851 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:29.493208 (+ 357us) server_negotiation.cc:299] Negotiation successful
0926 20:44:29.493463 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:44:29.494674 8665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:29.604483 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 8542
I20260926 20:44:29.624212 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 8676
Sep 26 20:44:29 dist-test-slave-4d5t krb5kdc[8436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:44:29.643236 8956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.625415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58544 (local address 127.23.44.194:45411)
0926 20:44:29.625623 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:29.625628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:29.625647 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:44:29.625717 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:29.625721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:29.625796 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:29.625890 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:29.627233 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.628097 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.629641 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.630084 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.630214 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.635154 (+ 4940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:29.635173 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:29.635192 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:29.635243 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:29.639816 (+ 4573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:29.640734 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:29.640742 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:29.640744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:29.640891 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:29.641515 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:29.641518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:29.641520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:29.641980 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:29.642256 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:29.642645 (+ 389us) server_negotiation.cc:299] Negotiation successful
0926 20:44:29.642843 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260926 20:44:29.643980 8799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:29.739190 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 8676
I20260926 20:44:29.756902 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 8810
Sep 26 20:44:29 dist-test-slave-4d5t krb5kdc[8436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:44:29.775177 8952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:29.757848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58262 (local address 127.23.44.195:41871)
0926 20:44:29.758100 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:29.758107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:29.758129 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:44:29.758267 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:29.758274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:29.758356 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:29.758500 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:29.760085 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.761000 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.762375 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:29.762761 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:29.762897 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:29.768032 (+ 5135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:29.768052 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:29.768069 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:29.768119 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:29.771506 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:29.772355 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:29.772363 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:29.772366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:29.772528 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:29.772981 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:29.772987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:29.772991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:29.773392 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:29.773698 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:29.774258 (+ 560us) server_negotiation.cc:299] Negotiation successful
0926 20:44:29.774624 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260926 20:44:29.776156 8933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:29.867954 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 8810
I20260926 20:44:29.886158 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 8449
I20260926 20:44:29.887501 8511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:29.999112 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 8449
2026-09-26T20:44:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4657 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T20:44:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:44:30Z Disabled control of system clock
I20260926 20:44:30.082996 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33141
--webserver_interface=127.23.44.254
--webserver_port=0
--builtin_ntp_servers=127.23.44.212:43841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:33141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 20:44:30.463821 8991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:30.464252 8991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:30.464442 8991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:30.474470 8991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:44:30.474627 8991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:30.474676 8991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:30.474749 8991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:44:30.474810 8991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:44:30.485085 8991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:43841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:33141
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:30.486402 8991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:30.488210 8991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:30.501307 8999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:30.501228 8997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:30.501384 8996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:30.503509 8991 server_base.cc:1034] running on GCE node
I20260926 20:44:30.504655 8991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:30.506832 8991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:30.508164 8991 hybrid_clock.cc:648] HybridClock initialized: now 1790455470508077 us; error 68 us; skew 500 ppm
I20260926 20:44:30.508741 8991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:44:30.511720 8991 webserver.cc:460] Webserver started at http://127.23.44.254:46245/ using document root <none> and password file <none>
I20260926 20:44:30.512537 8991 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:30.512737 8991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:30.513140 8991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:30.516546 8991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "d2866b2bcc114fe38e87ef4064e059bd"
format_stamp: "Formatted at 2026-09-26 20:44:30 on dist-test-slave-4d5t"
server_key: "7ebe9c1e6e61e7c23361a9e6384f1725"
server_key_iv: "468fadb5dfc2d15d2aee62a905027492"
server_key_version: "encryptionkey@0"
I20260926 20:44:30.517566 8991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "d2866b2bcc114fe38e87ef4064e059bd"
format_stamp: "Formatted at 2026-09-26 20:44:30 on dist-test-slave-4d5t"
server_key: "7ebe9c1e6e61e7c23361a9e6384f1725"
server_key_iv: "468fadb5dfc2d15d2aee62a905027492"
server_key_version: "encryptionkey@0"
I20260926 20:44:30.524469 8991 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260926 20:44:30.529057 9005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:30.530577 8991 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:44:30.530887 8991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "d2866b2bcc114fe38e87ef4064e059bd"
format_stamp: "Formatted at 2026-09-26 20:44:30 on dist-test-slave-4d5t"
server_key: "7ebe9c1e6e61e7c23361a9e6384f1725"
server_key_iv: "468fadb5dfc2d15d2aee62a905027492"
server_key_version: "encryptionkey@0"
I20260926 20:44:30.531111 8991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:30.571758 8991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:30.573047 8991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:30.573438 8991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:30.596385 8991 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:33141
I20260926 20:44:30.596426 9056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:33141 every 8 connection(s)
I20260926 20:44:30.597898 8991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:44:30.602062 9057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:30.606235 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 8991
I20260926 20:44:30.606449 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:44:30.607086 23731 external_mini_cluster.cc:1428] Setting key 5494b634444bcde8194b83cc12653d0f
I20260926 20:44:30.617405 9057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd: Bootstrap starting.
I20260926 20:44:30.621856 9057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:30.622784 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:30.609552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58358 (local address 127.23.44.254:33141)
0926 20:44:30.610590 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:30.610601 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:30.610643 (+ 42us) server_negotiation.cc:407] Connection header received
0926 20:44:30.612299 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:30.612324 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:30.612914 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:30.613275 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:30.613942 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:30.615259 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:30.617622 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:30.618395 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:30.618541 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:30.620084 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:30.620094 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:30.620109 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:30.620770 (+ 661us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:30.620867 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:30.620890 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:30.620989 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:30.621356 (+ 367us) server_negotiation.cc:299] Negotiation successful
0926 20:44:30.621739 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":421,"threads_started":1}
I20260926 20:44:30.623138 9057 log.cc:824] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:30.627331 9057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd: No bootstrap required, opened a new log
I20260926 20:44:30.637828 9057 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2866b2bcc114fe38e87ef4064e059bd" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33141 } }
I20260926 20:44:30.638356 9057 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:30.638500 9057 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2866b2bcc114fe38e87ef4064e059bd, State: Initialized, Role: FOLLOWER
I20260926 20:44:30.639154 9057 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2866b2bcc114fe38e87ef4064e059bd" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33141 } }
I20260926 20:44:30.639451 9057 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:44:30.639619 9057 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:44:30.639822 9057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:30.642481 9057 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2866b2bcc114fe38e87ef4064e059bd" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33141 } }
I20260926 20:44:30.643090 9057 leader_election.cc:302] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2866b2bcc114fe38e87ef4064e059bd; no voters:
I20260926 20:44:30.643826 9057 leader_election.cc:288] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:44:30.643953 9062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:30.644941 9062 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [term 1 LEADER]: Becoming Leader. State: Replica: d2866b2bcc114fe38e87ef4064e059bd, State: Running, Role: LEADER
I20260926 20:44:30.645530 9062 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2866b2bcc114fe38e87ef4064e059bd" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33141 } }
I20260926 20:44:30.646317 9057 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:44:30.649127 9064 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2866b2bcc114fe38e87ef4064e059bd. Latest consensus state: current_term: 1 leader_uuid: "d2866b2bcc114fe38e87ef4064e059bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2866b2bcc114fe38e87ef4064e059bd" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33141 } } }
I20260926 20:44:30.649652 9064 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:30.654767 9063 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2866b2bcc114fe38e87ef4064e059bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2866b2bcc114fe38e87ef4064e059bd" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33141 } } }
I20260926 20:44:30.656592 9063 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd [sys.catalog]: This master's current role is: LEADER
W20260926 20:44:30.661698 9078 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:44:30.661857 9078 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:44:30.662010 9071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:44:30.664434 9071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:44:30.675535 9071 catalog_manager.cc:1347] Generated new cluster ID: 9830b3b6da7f4f1f81efd2417b60d14d
I20260926 20:44:30.675675 9071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:44:30.695290 9071 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:44:30.696638 9071 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:44:30.713889 9071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd: Generated new TSK 0
I20260926 20:44:30.714885 9071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:44:30.733443 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33141
--builtin_ntp_servers=127.23.44.212:43841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 20:44:31.110409 9081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:31.110708 9081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:31.110846 9081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:31.118597 9081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:31.118747 9081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:31.118932 9081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:44:31.131374 9081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:43841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:31.133121 9081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:31.135190 9081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:31.150902 9086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:31.150870 9087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:31.150851 9089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:31.152405 9081 server_base.cc:1034] running on GCE node
I20260926 20:44:31.153162 9081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:31.154273 9081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:31.155531 9081 hybrid_clock.cc:648] HybridClock initialized: now 1790455471155467 us; error 53 us; skew 500 ppm
I20260926 20:44:31.155941 9081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:44:31.158840 9081 webserver.cc:460] Webserver started at http://127.23.44.193:46249/ using document root <none> and password file <none>
I20260926 20:44:31.159605 9081 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:31.159701 9081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:31.160074 9081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:31.163347 9081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "99deb8a61d8345289aa383f87b53d0a8"
format_stamp: "Formatted at 2026-09-26 20:44:31 on dist-test-slave-4d5t"
server_key: "8a7b9d2177c410071778b53977e8b9b5"
server_key_iv: "77f3eec4776d47fca309fa7634144799"
server_key_version: "encryptionkey@0"
I20260926 20:44:31.164230 9081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "99deb8a61d8345289aa383f87b53d0a8"
format_stamp: "Formatted at 2026-09-26 20:44:31 on dist-test-slave-4d5t"
server_key: "8a7b9d2177c410071778b53977e8b9b5"
server_key_iv: "77f3eec4776d47fca309fa7634144799"
server_key_version: "encryptionkey@0"
I20260926 20:44:31.169939 9081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260926 20:44:31.174295 9095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:31.176105 9081 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:44:31.176339 9081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "99deb8a61d8345289aa383f87b53d0a8"
format_stamp: "Formatted at 2026-09-26 20:44:31 on dist-test-slave-4d5t"
server_key: "8a7b9d2177c410071778b53977e8b9b5"
server_key_iv: "77f3eec4776d47fca309fa7634144799"
server_key_version: "encryptionkey@0"
I20260926 20:44:31.176658 9081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:31.212710 9081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:31.213653 9081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:31.214043 9081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:31.215088 9081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:31.216763 9081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:31.216851 9081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:31.216948 9081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:31.217005 9081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:31.254810 9081 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:35389
I20260926 20:44:31.254964 9207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:35389 every 8 connection(s)
I20260926 20:44:31.256209 9081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:44:31.263624 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 9081
I20260926 20:44:31.263827 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:44:31.264413 23731 external_mini_cluster.cc:1428] Setting key a051b70b5dee3a2d3d529f135dc2939f
I20260926 20:44:31.267561 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33141
--builtin_ntp_servers=127.23.44.212:43841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 20:44:31.273097 9210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:31.259715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:50077 (local address 127.23.44.254:33141)
0926 20:44:31.260246 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:31.260253 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:31.262070 (+ 1817us) server_negotiation.cc:407] Connection header received
0926 20:44:31.264186 (+ 2116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:31.264192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:31.264301 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:31.264441 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:31.266075 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:31.267184 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:31.269798 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:31.270412 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:31.270591 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:31.272167 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:31.272171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:31.272174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:31.272195 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:31.272235 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:31.272248 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:31.272349 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:31.272621 (+ 272us) server_negotiation.cc:299] Negotiation successful
0926 20:44:31.272726 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":268,"threads_started":1}
I20260926 20:44:31.274194 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:31.260389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33141 (local address 127.23.44.193:50077)
0926 20:44:31.261886 (+ 1497us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:31.261933 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:31.263849 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:31.264746 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:31.264755 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:31.264854 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:31.265638 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:31.265658 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:31.267431 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:31.267437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:31.269539 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:31.269574 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:31.270722 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:31.270732 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:31.271017 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:31.271804 (+ 787us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:44:31.271806 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:31.271850 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:44:31.271937 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:31.272348 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:31.272362 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:31.272373 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 20:44:31.272591 (+ 218us) client_negotiation.cc:240] Negotiation successful
0926 20:44:31.273046 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"thread_start_us":433,"threads_started":1}
I20260926 20:44:31.275897 9208 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33141
I20260926 20:44:31.276211 9208 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:31.277067 9208 heartbeater.cc:511] Master 127.23.44.254:33141 requested a full tablet report, sending...
I20260926 20:44:31.279488 9022 ts_manager.cc:194] Registered new tserver with Master: 99deb8a61d8345289aa383f87b53d0a8 (127.23.44.193:35389)
I20260926 20:44:31.281242 9022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.44.193:50077
W20260926 20:44:31.639837 9212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:31.640192 9212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:31.640353 9212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:31.648924 9212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:31.649206 9212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:31.649410 9212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:44:31.659281 9212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:43841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:31.660732 9212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:31.662647 9212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:31.678236 9218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:31.678239 9217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:31.678236 9220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:31.682331 9212 server_base.cc:1034] running on GCE node
I20260926 20:44:31.683153 9212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:31.684478 9212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:31.685959 9212 hybrid_clock.cc:648] HybridClock initialized: now 1790455471685921 us; error 92 us; skew 500 ppm
I20260926 20:44:31.686347 9212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:44:31.689060 9212 webserver.cc:460] Webserver started at http://127.23.44.194:45763/ using document root <none> and password file <none>
I20260926 20:44:31.689810 9212 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:31.689906 9212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:31.690222 9212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:31.693624 9212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "0f22f0f06aa24f1784450ede580b35eb"
format_stamp: "Formatted at 2026-09-26 20:44:31 on dist-test-slave-4d5t"
server_key: "4d1fe3bee9a6c78c8d5fc6700437ee8b"
server_key_iv: "7f65a2fdb3aef44f7f00afac14c28548"
server_key_version: "encryptionkey@0"
I20260926 20:44:31.694525 9212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "0f22f0f06aa24f1784450ede580b35eb"
format_stamp: "Formatted at 2026-09-26 20:44:31 on dist-test-slave-4d5t"
server_key: "4d1fe3bee9a6c78c8d5fc6700437ee8b"
server_key_iv: "7f65a2fdb3aef44f7f00afac14c28548"
server_key_version: "encryptionkey@0"
I20260926 20:44:31.700932 9212 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260926 20:44:31.704679 9226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:31.706086 9212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:44:31.706290 9212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "0f22f0f06aa24f1784450ede580b35eb"
format_stamp: "Formatted at 2026-09-26 20:44:31 on dist-test-slave-4d5t"
server_key: "4d1fe3bee9a6c78c8d5fc6700437ee8b"
server_key_iv: "7f65a2fdb3aef44f7f00afac14c28548"
server_key_version: "encryptionkey@0"
I20260926 20:44:31.706532 9212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:31.727890 9212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:31.728947 9212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:31.729332 9212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:31.730552 9212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:31.732295 9212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:31.732398 9212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:31.732600 9212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:31.732686 9212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260926 20:44:31.772085 9212 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:35503
I20260926 20:44:31.772154 9338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:35503 every 8 connection(s)
I20260926 20:44:31.773720 9212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:44:31.779953 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 9212
I20260926 20:44:31.780143 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:44:31.780678 23731 external_mini_cluster.cc:1428] Setting key 6735c994c38ceda6a775ec5a2e1dc4a1
I20260926 20:44:31.783529 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33141
--builtin_ntp_servers=127.23.44.212:43841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 20:44:31.788061 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:31.776484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:52959 (local address 127.23.44.254:33141)
0926 20:44:31.776982 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:31.776986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:31.778079 (+ 1093us) server_negotiation.cc:407] Connection header received
0926 20:44:31.780409 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:31.780414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:31.780494 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:31.780718 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:31.782086 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:31.782943 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:31.784883 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:31.785329 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:31.785485 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:31.787081 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:31.787086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:31.787090 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:31.787112 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:31.787166 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:31.787180 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:31.787289 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:31.787533 (+ 244us) server_negotiation.cc:299] Negotiation successful
0926 20:44:31.787647 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":300,"threads_started":1}
I20260926 20:44:31.789219 9342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:31.777051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33141 (local address 127.23.44.194:52959)
0926 20:44:31.777921 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:31.777962 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:31.779992 (+ 2030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:31.781000 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:31.781009 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:31.781151 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:31.781866 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:31.781909 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:31.783113 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:31.783120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:31.784703 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:31.784719 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:31.785599 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:31.785608 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:31.785901 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:31.786700 (+ 799us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:44:31.786703 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:31.786746 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:44:31.786844 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:31.787288 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:31.787302 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:31.787313 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 20:44:31.787525 (+ 212us) client_negotiation.cc:240] Negotiation successful
0926 20:44:31.787985 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":269,"threads_started":1}
I20260926 20:44:31.790794 9339 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33141
I20260926 20:44:31.791146 9339 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:31.792081 9339 heartbeater.cc:511] Master 127.23.44.254:33141 requested a full tablet report, sending...
I20260926 20:44:31.793766 9022 ts_manager.cc:194] Registered new tserver with Master: 0f22f0f06aa24f1784450ede580b35eb (127.23.44.194:35503)
I20260926 20:44:31.794581 9022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.44.194:52959
W20260926 20:44:32.137771 9343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:32.138274 9343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:32.138509 9343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:32.147838 9343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:32.147998 9343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:32.148185 9343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:44:32.158301 9343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:43841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:32.160058 9343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:32.162678 9343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:32.178689 9348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:32.178694 9349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:32.179170 9351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:32.179857 9343 server_base.cc:1034] running on GCE node
I20260926 20:44:32.180658 9343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:32.181802 9343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:32.183029 9343 hybrid_clock.cc:648] HybridClock initialized: now 1790455472182944 us; error 83 us; skew 500 ppm
I20260926 20:44:32.183367 9343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:44:32.185971 9343 webserver.cc:460] Webserver started at http://127.23.44.195:34197/ using document root <none> and password file <none>
I20260926 20:44:32.186712 9343 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:32.186806 9343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:32.187075 9343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:32.190481 9343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "3c6f2d85c6ee48c7844d66a623e3f53b"
format_stamp: "Formatted at 2026-09-26 20:44:32 on dist-test-slave-4d5t"
server_key: "ecb98a65e1a9f07e6a47d07977c1e213"
server_key_iv: "2724d367175e450c23a83a98b68fbd77"
server_key_version: "encryptionkey@0"
I20260926 20:44:32.191425 9343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "3c6f2d85c6ee48c7844d66a623e3f53b"
format_stamp: "Formatted at 2026-09-26 20:44:32 on dist-test-slave-4d5t"
server_key: "ecb98a65e1a9f07e6a47d07977c1e213"
server_key_iv: "2724d367175e450c23a83a98b68fbd77"
server_key_version: "encryptionkey@0"
I20260926 20:44:32.197901 9343 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260926 20:44:32.201884 9357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:32.203570 9343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:44:32.203740 9343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "3c6f2d85c6ee48c7844d66a623e3f53b"
format_stamp: "Formatted at 2026-09-26 20:44:32 on dist-test-slave-4d5t"
server_key: "ecb98a65e1a9f07e6a47d07977c1e213"
server_key_iv: "2724d367175e450c23a83a98b68fbd77"
server_key_version: "encryptionkey@0"
I20260926 20:44:32.204021 9343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:32.219347 9343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:32.220312 9343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:32.220722 9343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:32.221920 9343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:32.223577 9343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:32.223673 9343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:32.223812 9343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:32.223886 9343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:32.263651 9343 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:33093
I20260926 20:44:32.263832 9469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:33093 every 8 connection(s)
I20260926 20:44:32.265177 9343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:44:32.273180 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 9343
I20260926 20:44:32.273389 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:44:32.273995 23731 external_mini_cluster.cc:1428] Setting key c693a04fcb83da54406dfa535debc839
I20260926 20:44:32.283007 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.268515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:49807 (local address 127.23.44.254:33141)
0926 20:44:32.268768 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:32.268778 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:32.270756 (+ 1978us) server_negotiation.cc:407] Connection header received
0926 20:44:32.273982 (+ 3226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:32.273988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:32.274092 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:32.274197 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:32.275738 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.276691 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.279451 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.279878 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.279991 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.282118 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:32.282124 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:32.282135 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:32.282158 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:32.282194 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:32.282208 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:32.282378 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:32.282636 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 20:44:32.282734 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:44:32.284197 9472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.269197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33141 (local address 127.23.44.195:49807)
0926 20:44:32.270548 (+ 1351us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:32.270594 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:32.273588 (+ 2994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:32.274505 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:32.274521 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:32.274675 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:32.275557 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:32.275583 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.277011 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.277018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:32.279239 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:32.279268 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.280134 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.280141 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:32.280429 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.281670 (+ 1241us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:44:32.281673 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:32.281739 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:44:32.281836 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:32.282381 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:32.282389 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:32.282395 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 20:44:32.282618 (+ 223us) client_negotiation.cc:240] Negotiation successful
0926 20:44:32.283004 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":488,"threads_started":1}
I20260926 20:44:32.284884 9208 heartbeater.cc:503] Master 127.23.44.254:33141 was elected leader, sending a full tablet report...
I20260926 20:44:32.285979 9470 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33141
I20260926 20:44:32.286302 9470 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:32.287150 9470 heartbeater.cc:511] Master 127.23.44.254:33141 requested a full tablet report, sending...
I20260926 20:44:32.288748 9022 ts_manager.cc:194] Registered new tserver with Master: 3c6f2d85c6ee48c7844d66a623e3f53b (127.23.44.195:33093)
I20260926 20:44:32.290002 9022 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.44.195:49807
I20260926 20:44:32.292366 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:44:32.308190 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.300879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58370 (local address 127.23.44.254:33141)
0926 20:44:32.301104 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:32.301114 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:32.301657 (+ 543us) server_negotiation.cc:407] Connection header received
0926 20:44:32.301806 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:32.301810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:32.301883 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:32.301969 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:32.302702 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.303436 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.305170 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.305650 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.305776 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.307198 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:32.307204 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:32.307208 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:32.307232 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:32.307279 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:32.307295 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:32.307545 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:32.307638 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 20:44:32.307747 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:44:32.313391 9022 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:44:32.322453 9022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:44:32.351470 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.337216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:35389 (local address 127.0.0.1:49198)
0926 20:44:32.338349 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:32.338401 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:32.338633 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:32.339916 (+ 1283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:32.339932 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:32.339975 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:32.340471 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:32.340487 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.343173 (+ 2686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.343182 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:32.346565 (+ 3383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:32.346594 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.350017 (+ 3423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.350027 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:32.350399 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.350478 (+ 79us) client_negotiation.cc:769] Sending connection context
0926 20:44:32.350656 (+ 178us) client_negotiation.cc:240] Negotiation successful
0926 20:44:32.350955 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"thread_start_us":993,"threads_started":1}
I20260926 20:44:32.352806 9483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.337509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49198 (local address 127.23.44.193:35389)
0926 20:44:32.338174 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:32.338184 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:32.338872 (+ 688us) server_negotiation.cc:407] Connection header received
0926 20:44:32.338998 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:32.339010 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:32.339357 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:32.339570 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:32.340724 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.342917 (+ 2193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.346938 (+ 4021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.349215 (+ 2277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.349426 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.349830 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:32.351972 (+ 2142us) server_negotiation.cc:299] Negotiation successful
0926 20:44:32.352263 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":341,"threads_started":1}
I20260926 20:44:32.358891 9143 tablet_service.cc:1467] Processing CreateTablet for tablet 45777ad3e16c4883a48300a237bd78e4 (DEFAULT_TABLE table=test-table [id=4f0a609f2f47405ea58d9ad87d682989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:32.360563 9143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45777ad3e16c4883a48300a237bd78e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:32.361434 9484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.338156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:33093 (local address 127.0.0.1:44694)
0926 20:44:32.353175 (+ 15019us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:32.353197 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:32.353362 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:32.355100 (+ 1738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:32.355106 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:32.355137 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:32.355479 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:32.355487 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.356923 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.356931 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:32.358768 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:32.358784 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.360441 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.360448 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:32.360692 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.360727 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 20:44:32.360835 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 20:44:32.360991 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14849,"spinlock_wait_cycles":15872,"thread_start_us":15290,"threads_started":1}
I20260926 20:44:32.361995 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.339497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:35503 (local address 127.0.0.1:57730)
0926 20:44:32.353208 (+ 13711us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:32.353250 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:32.353362 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:32.354839 (+ 1477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:32.354844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:32.354879 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:32.355068 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:32.355079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.357186 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.357192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:32.359295 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:32.359324 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.361168 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.361174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:32.361422 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.361466 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 20:44:32.361531 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 20:44:32.361672 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13604,"spinlock_wait_cycles":7680,"thread_start_us":14601,"threads_started":1}
I20260926 20:44:32.362615 9485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.338507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44694 (local address 127.23.44.195:33093)
0926 20:44:32.354403 (+ 15896us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:32.354410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:32.354437 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:44:32.354517 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:32.354523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:32.354725 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:32.354885 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:32.355672 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.356761 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.358964 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.360217 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.360362 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.361965 (+ 1603us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:32.362046 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 20:44:32.362222 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15741,"thread_start_us":15586,"threads_started":1}
I20260926 20:44:32.363248 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.340664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.23.44.194:35503)
0926 20:44:32.354099 (+ 13435us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:32.354111 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:32.354149 (+ 38us) server_negotiation.cc:407] Connection header received
0926 20:44:32.354274 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:32.354283 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:32.354577 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:32.354831 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:32.355270 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.356988 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.359492 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.360979 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.362374 (+ 1395us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.362627 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:32.362728 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 20:44:32.362908 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13243,"thread_start_us":13158,"threads_started":1}
I20260926 20:44:32.366679 9274 tablet_service.cc:1467] Processing CreateTablet for tablet 45777ad3e16c4883a48300a237bd78e4 (DEFAULT_TABLE table=test-table [id=4f0a609f2f47405ea58d9ad87d682989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:32.366782 9405 tablet_service.cc:1467] Processing CreateTablet for tablet 45777ad3e16c4883a48300a237bd78e4 (DEFAULT_TABLE table=test-table [id=4f0a609f2f47405ea58d9ad87d682989]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:32.368011 9274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45777ad3e16c4883a48300a237bd78e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:32.368319 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45777ad3e16c4883a48300a237bd78e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:32.378290 9488 tablet_bootstrap.cc:492] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8: Bootstrap starting.
I20260926 20:44:32.381358 9489 tablet_bootstrap.cc:492] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb: Bootstrap starting.
I20260926 20:44:32.382323 9488 tablet_bootstrap.cc:654] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:32.383503 9488 log.cc:824] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:32.385928 9489 tablet_bootstrap.cc:654] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:32.386885 9490 tablet_bootstrap.cc:492] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b: Bootstrap starting.
I20260926 20:44:32.387346 9489 log.cc:824] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:32.387981 9488 tablet_bootstrap.cc:492] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8: No bootstrap required, opened a new log
I20260926 20:44:32.388348 9488 ts_tablet_manager.cc:1397] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.004s
I20260926 20:44:32.389971 9489 tablet_bootstrap.cc:492] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb: No bootstrap required, opened a new log
I20260926 20:44:32.390259 9489 ts_tablet_manager.cc:1397] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 20:44:32.390581 9490 tablet_bootstrap.cc:654] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:32.391825 9490 log.cc:824] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:32.396096 9490 tablet_bootstrap.cc:492] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b: No bootstrap required, opened a new log
I20260926 20:44:32.396435 9490 ts_tablet_manager.cc:1397] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260926 20:44:32.397580 9488 raft_consensus.cc:348] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99deb8a61d8345289aa383f87b53d0a8" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35389 } } peers { permanent_uuid: "0f22f0f06aa24f1784450ede580b35eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35503 } } peers { permanent_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33093 } }
I20260926 20:44:32.398029 9488 raft_consensus.cc:374] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:32.398170 9488 raft_consensus.cc:729] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99deb8a61d8345289aa383f87b53d0a8, State: Initialized, Role: FOLLOWER
I20260926 20:44:32.398829 9488 consensus_queue.cc:260] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99deb8a61d8345289aa383f87b53d0a8" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35389 } } peers { permanent_uuid: "0f22f0f06aa24f1784450ede580b35eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35503 } } peers { permanent_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33093 } }
I20260926 20:44:32.400612 9488 ts_tablet_manager.cc:1428] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260926 20:44:32.401839 9489 raft_consensus.cc:348] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99deb8a61d8345289aa383f87b53d0a8" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35389 } } peers { permanent_uuid: "0f22f0f06aa24f1784450ede580b35eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35503 } } peers { permanent_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33093 } }
I20260926 20:44:32.402336 9489 raft_consensus.cc:374] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:32.402436 9489 raft_consensus.cc:729] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f22f0f06aa24f1784450ede580b35eb, State: Initialized, Role: FOLLOWER
I20260926 20:44:32.403064 9489 consensus_queue.cc:260] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99deb8a61d8345289aa383f87b53d0a8" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35389 } } peers { permanent_uuid: "0f22f0f06aa24f1784450ede580b35eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35503 } } peers { permanent_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33093 } }
I20260926 20:44:32.405790 9339 heartbeater.cc:503] Master 127.23.44.254:33141 was elected leader, sending a full tablet report...
I20260926 20:44:32.406440 9489 ts_tablet_manager.cc:1428] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260926 20:44:32.407763 9490 raft_consensus.cc:348] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99deb8a61d8345289aa383f87b53d0a8" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35389 } } peers { permanent_uuid: "0f22f0f06aa24f1784450ede580b35eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35503 } } peers { permanent_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33093 } }
I20260926 20:44:32.408267 9490 raft_consensus.cc:374] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:32.408392 9490 raft_consensus.cc:729] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c6f2d85c6ee48c7844d66a623e3f53b, State: Initialized, Role: FOLLOWER
I20260926 20:44:32.409045 9490 consensus_queue.cc:260] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99deb8a61d8345289aa383f87b53d0a8" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35389 } } peers { permanent_uuid: "0f22f0f06aa24f1784450ede580b35eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35503 } } peers { permanent_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33093 } }
I20260926 20:44:32.410276 9470 heartbeater.cc:503] Master 127.23.44.254:33141 was elected leader, sending a full tablet report...
I20260926 20:44:32.410914 9490 ts_tablet_manager.cc:1428] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b: Time spent starting tablet: real 0.014s user 0.008s sys 0.007s
W20260926 20:44:32.509196 9209 tablet.cc:2369] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:44:32.517352 9471 tablet.cc:2369] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:44:32.525975 9340 tablet.cc:2369] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:44:32.684147 9496 raft_consensus.cc:482] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:44:32.684496 9496 raft_consensus.cc:504] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99deb8a61d8345289aa383f87b53d0a8" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35389 } } peers { permanent_uuid: "0f22f0f06aa24f1784450ede580b35eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35503 } } peers { permanent_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33093 } }
I20260926 20:44:32.686364 9496 leader_election.cc:288] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99deb8a61d8345289aa383f87b53d0a8 (127.23.44.193:35389), 0f22f0f06aa24f1784450ede580b35eb (127.23.44.194:35503)
I20260926 20:44:32.694387 9483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.686887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:40303 (local address 127.23.44.193:35389)
0926 20:44:32.687124 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:32.687128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:32.687170 (+ 42us) server_negotiation.cc:407] Connection header received
0926 20:44:32.687251 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:32.687257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:32.687354 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:32.687999 (+ 645us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:32.688140 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.689033 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.691756 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.693444 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.693630 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.693712 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:32.694018 (+ 306us) server_negotiation.cc:299] Negotiation successful
0926 20:44:32.694153 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 20:44:32.694528 9472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.686638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:35389 (local address 127.23.44.195:40303)
0926 20:44:32.686811 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:32.686828 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:32.686972 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:32.687576 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:32.687581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:32.687616 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:32.687830 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:32.687838 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.689184 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.689188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:32.691454 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:32.691471 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.693625 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.693630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:32.693802 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.693829 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 20:44:32.693986 (+ 157us) client_negotiation.cc:240] Negotiation successful
0926 20:44:32.694159 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":28}
I20260926 20:44:32.694761 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.687439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:38983 (local address 127.23.44.194:35503)
0926 20:44:32.687633 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:32.687650 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:32.687939 (+ 289us) server_negotiation.cc:407] Connection header received
0926 20:44:32.688047 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:32.688051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:32.688135 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:32.688244 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:32.688891 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.689649 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.692260 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.693802 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.693965 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.694029 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:32.694384 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 20:44:32.694541 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:44:32.694896 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.687128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:35503 (local address 127.23.44.195:38983)
0926 20:44:32.687828 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:32.687842 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:32.687945 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:32.688279 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:32.688283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:32.688304 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:32.688703 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:32.688721 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.689816 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.689822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:32.692001 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:32.692011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.693983 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.693989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:32.694171 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.694206 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 20:44:32.694323 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 20:44:32.694462 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":1,"thread_start_us":358,"threads_started":1}
I20260926 20:44:32.696043 9163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45777ad3e16c4883a48300a237bd78e4" candidate_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99deb8a61d8345289aa383f87b53d0a8" is_pre_election: true
I20260926 20:44:32.696043 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45777ad3e16c4883a48300a237bd78e4" candidate_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f22f0f06aa24f1784450ede580b35eb" is_pre_election: true
I20260926 20:44:32.696689 9294 raft_consensus.cc:2454] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c6f2d85c6ee48c7844d66a623e3f53b in term 0.
I20260926 20:44:32.696681 9163 raft_consensus.cc:2454] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c6f2d85c6ee48c7844d66a623e3f53b in term 0.
I20260926 20:44:32.697471 9360 leader_election.cc:302] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f22f0f06aa24f1784450ede580b35eb, 3c6f2d85c6ee48c7844d66a623e3f53b; no voters:
I20260926 20:44:32.697984 9496 raft_consensus.cc:2781] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:44:32.698164 9496 raft_consensus.cc:482] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:44:32.698283 9496 raft_consensus.cc:3037] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:32.701260 9496 raft_consensus.cc:504] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99deb8a61d8345289aa383f87b53d0a8" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35389 } } peers { permanent_uuid: "0f22f0f06aa24f1784450ede580b35eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35503 } } peers { permanent_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33093 } }
I20260926 20:44:32.702311 9496 leader_election.cc:288] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [CANDIDATE]: Term 1 election: Requested vote from peers 99deb8a61d8345289aa383f87b53d0a8 (127.23.44.193:35389), 0f22f0f06aa24f1784450ede580b35eb (127.23.44.194:35503)
I20260926 20:44:32.703034 9163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45777ad3e16c4883a48300a237bd78e4" candidate_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99deb8a61d8345289aa383f87b53d0a8"
I20260926 20:44:32.703275 9163 raft_consensus.cc:3037] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:32.703263 9294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45777ad3e16c4883a48300a237bd78e4" candidate_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f22f0f06aa24f1784450ede580b35eb"
I20260926 20:44:32.703604 9294 raft_consensus.cc:3037] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:32.705709 9163 raft_consensus.cc:2454] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c6f2d85c6ee48c7844d66a623e3f53b in term 1.
I20260926 20:44:32.706243 9360 leader_election.cc:302] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c6f2d85c6ee48c7844d66a623e3f53b, 99deb8a61d8345289aa383f87b53d0a8; no voters:
I20260926 20:44:32.706600 9496 raft_consensus.cc:2781] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:32.706782 9294 raft_consensus.cc:2454] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c6f2d85c6ee48c7844d66a623e3f53b in term 1.
I20260926 20:44:32.707131 9496 raft_consensus.cc:686] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [term 1 LEADER]: Becoming Leader. State: Replica: 3c6f2d85c6ee48c7844d66a623e3f53b, State: Running, Role: LEADER
I20260926 20:44:32.708101 9496 consensus_queue.cc:237] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99deb8a61d8345289aa383f87b53d0a8" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35389 } } peers { permanent_uuid: "0f22f0f06aa24f1784450ede580b35eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35503 } } peers { permanent_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33093 } }
I20260926 20:44:32.713236 9022 catalog_manager.cc:5495] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c6f2d85c6ee48c7844d66a623e3f53b (127.23.44.195). New cstate: current_term: 1 leader_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99deb8a61d8345289aa383f87b53d0a8" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f22f0f06aa24f1784450ede580b35eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c6f2d85c6ee48c7844d66a623e3f53b" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 33093 } health_report { overall_health: HEALTHY } } }
I20260926 20:44:32.735713 9485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.729691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44708 (local address 127.23.44.195:33093)
0926 20:44:32.729861 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:32.729866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:32.729883 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:44:32.730039 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:32.730043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:32.730122 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:32.730248 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:44:32.730697 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.731449 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.733195 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.733638 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.733757 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.734519 (+ 762us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:44:32.735177 (+ 658us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:44:32.735296 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:32.735434 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 20:44:32.735505 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 20:44:32.745177 9294 raft_consensus.cc:1261] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb [term 1 FOLLOWER]: Refusing update from remote peer 3c6f2d85c6ee48c7844d66a623e3f53b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:44:32.745177 9163 raft_consensus.cc:1261] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8 [term 1 FOLLOWER]: Refusing update from remote peer 3c6f2d85c6ee48c7844d66a623e3f53b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:44:32.746423 9498 consensus_queue.cc:1035] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "99deb8a61d8345289aa383f87b53d0a8" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:44:32.746863 9496 consensus_queue.cc:1035] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f22f0f06aa24f1784450ede580b35eb" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:44:32.762172 9503 mvcc.cc:205] Tried to move back new op lower bound from 7333705616351342592 to 7333705616216805376. Current Snapshot: MvccSnapshot[applied={T|T < 7333705616351342592}]
I20260926 20:44:32.766724 9502 mvcc.cc:205] Tried to move back new op lower bound from 7333705616351342592 to 7333705616216805376. Current Snapshot: MvccSnapshot[applied={T|T < 7333705616351342592}]
I20260926 20:44:32.782418 9021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:44:32.782727 9021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:44:32.787281 9021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2866b2bcc114fe38e87ef4064e059bd: Sending DeleteTablet for 3 replicas of tablet 45777ad3e16c4883a48300a237bd78e4
I20260926 20:44:32.788687 9274 tablet_service.cc:1514] Processing DeleteTablet for tablet 45777ad3e16c4883a48300a237bd78e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:32 UTC) from {username='slave'} at 127.0.0.1:57730
I20260926 20:44:32.788694 9405 tablet_service.cc:1514] Processing DeleteTablet for tablet 45777ad3e16c4883a48300a237bd78e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:32 UTC) from {username='slave'} at 127.0.0.1:44694
I20260926 20:44:32.788764 9143 tablet_service.cc:1514] Processing DeleteTablet for tablet 45777ad3e16c4883a48300a237bd78e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:32 UTC) from {username='slave'} at 127.0.0.1:49198
I20260926 20:44:32.791244 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 9081
I20260926 20:44:32.798928 9512 tablet_replica.cc:330] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8: stopping tablet replica
I20260926 20:44:32.799640 9512 raft_consensus.cc:2229] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:44:32.800325 9512 raft_consensus.cc:2258] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:32.801523 9510 tablet_replica.cc:330] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b: stopping tablet replica
I20260926 20:44:32.803102 9512 ts_tablet_manager.cc:1905] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:32.803653 9510 raft_consensus.cc:2229] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:44:32.811058 9510 raft_consensus.cc:2258] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:32.812202 9511 tablet_replica.cc:330] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb: stopping tablet replica
I20260926 20:44:32.812812 9511 raft_consensus.cc:2229] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:44:32.813380 9511 raft_consensus.cc:2258] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:32.813972 9510 ts_tablet_manager.cc:1905] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:32.814759 9512 ts_tablet_manager.cc:1918] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:32.814957 9512 log.cc:1196] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/45777ad3e16c4883a48300a237bd78e4
I20260926 20:44:32.815735 9512 ts_tablet_manager.cc:1939] T 45777ad3e16c4883a48300a237bd78e4 P 99deb8a61d8345289aa383f87b53d0a8: Deleting consensus metadata
I20260926 20:44:32.817986 9008 catalog_manager.cc:4841] TS 99deb8a61d8345289aa383f87b53d0a8 (127.23.44.193:35389): tablet 45777ad3e16c4883a48300a237bd78e4 (table test-table [id=4f0a609f2f47405ea58d9ad87d682989]) successfully deleted
I20260926 20:44:32.821465 9510 ts_tablet_manager.cc:1918] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:32.821653 9510 log.cc:1196] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/45777ad3e16c4883a48300a237bd78e4
I20260926 20:44:32.822206 9510 ts_tablet_manager.cc:1939] T 45777ad3e16c4883a48300a237bd78e4 P 3c6f2d85c6ee48c7844d66a623e3f53b: Deleting consensus metadata
I20260926 20:44:32.824007 9009 catalog_manager.cc:4841] TS 3c6f2d85c6ee48c7844d66a623e3f53b (127.23.44.195:33093): tablet 45777ad3e16c4883a48300a237bd78e4 (table test-table [id=4f0a609f2f47405ea58d9ad87d682989]) successfully deleted
I20260926 20:44:32.826066 9511 ts_tablet_manager.cc:1905] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:32.827287 9483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.793305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.23.44.193:35389)
0926 20:44:32.793552 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:32.793561 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:32.810543 (+ 16982us) server_negotiation.cc:407] Connection header received
0926 20:44:32.816791 (+ 6248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:32.816799 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:32.816948 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:32.817042 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:32.820152 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.821366 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.823434 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.824068 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.824175 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.825787 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:32.825796 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:32.825809 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:32.826457 (+ 648us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:32.826505 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:32.826525 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:32.826651 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:32.826788 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 20:44:32.826912 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:44:32.827945 9203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:32.834233 9511 ts_tablet_manager.cc:1918] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:32.834432 9511 log.cc:1196] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/45777ad3e16c4883a48300a237bd78e4
I20260926 20:44:32.835021 9511 ts_tablet_manager.cc:1939] T 45777ad3e16c4883a48300a237bd78e4 P 0f22f0f06aa24f1784450ede580b35eb: Deleting consensus metadata
I20260926 20:44:32.837204 9008 catalog_manager.cc:4841] TS 0f22f0f06aa24f1784450ede580b35eb (127.23.44.194:35503): tablet 45777ad3e16c4883a48300a237bd78e4 (table test-table [id=4f0a609f2f47405ea58d9ad87d682989]) successfully deleted
I20260926 20:44:32.922362 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 9081
I20260926 20:44:32.941587 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 9212
I20260926 20:44:32.950842 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:32.942448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57736 (local address 127.23.44.194:35503)
0926 20:44:32.942689 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:32.942696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:32.942722 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:44:32.942810 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:32.942816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:32.942935 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:32.943055 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:32.943438 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.944731 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.946183 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:32.946990 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:32.947132 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:32.948383 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:32.948395 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:32.948418 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:32.949382 (+ 964us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:32.949452 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:32.949481 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:32.949607 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:32.949858 (+ 251us) server_negotiation.cc:299] Negotiation successful
0926 20:44:32.949993 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 20:44:32.951696 9334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:33.047144 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 9212
I20260926 20:44:33.071368 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 9343
I20260926 20:44:33.080298 9485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:33.072257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44710 (local address 127.23.44.195:33093)
0926 20:44:33.072455 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:33.072463 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:33.072656 (+ 193us) server_negotiation.cc:407] Connection header received
0926 20:44:33.073173 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:33.073178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:33.073305 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:33.073433 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:33.073906 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:33.074859 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:33.076480 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:33.077110 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:33.077293 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:33.078701 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:33.078711 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:33.078728 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:33.079374 (+ 646us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:33.079429 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:33.079455 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:33.079596 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:33.079752 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 20:44:33.079872 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 20:44:33.081463 9465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:33.209501 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 9343
I20260926 20:44:33.232395 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 8991
I20260926 20:44:33.233708 9052 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:33.347961 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 8991
2026-09-26T20:44:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3349 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:44:33 dist-test-slave-4d5t krb5kdc[9520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:44:33 dist-test-slave-4d5t krb5kdc[9520](info): set up 2 sockets
Sep 26 20:44:33 dist-test-slave-4d5t krb5kdc[9520](info): commencing operation
krb5kdc: starting...
W20260926 20:44:34.509164 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:44:34 dist-test-slave-4d5t krb5kdc[9520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:44:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:44:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:44:34.800954 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33235
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:33235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:35.154534 9533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:35.154848 9533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:35.155009 9533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:35.162493 9533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:44:35.162613 9533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:35.162714 9533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:35.162829 9533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:44:35.162917 9533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:44:35.173241 9533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:33235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:33235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:35.174657 9533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:35.176016 9533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:35.188052 9541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:35.188133 9539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:35.188153 9538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:35.189337 9533 server_base.cc:1034] running on GCE node
I20260926 20:44:35.190207 9533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:35.191810 9533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:35.193254 9533 hybrid_clock.cc:648] HybridClock initialized: now 1790455475193195 us; error 60 us; skew 500 ppm
I20260926 20:44:35.193652 9533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:35 dist-test-slave-4d5t krb5kdc[9520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455475, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:35.197154 9533 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:44:35.198770 9533 webserver.cc:460] Webserver started at http://127.23.44.254:43543/ using document root <none> and password file <none>
I20260926 20:44:35.199465 9533 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:35.199572 9533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:35.199829 9533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:35.203105 9533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "a00afe2786d449b4afa84acecb04227b"
format_stamp: "Formatted at 2026-09-26 20:44:35 on dist-test-slave-4d5t"
server_key: "c5b47b8ccf426d9a7badd472f673d7f6"
server_key_iv: "c24737c6bb507187ee24939e7c8cb6d5"
server_key_version: "encryptionkey@0"
I20260926 20:44:35.203876 9533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "a00afe2786d449b4afa84acecb04227b"
format_stamp: "Formatted at 2026-09-26 20:44:35 on dist-test-slave-4d5t"
server_key: "c5b47b8ccf426d9a7badd472f673d7f6"
server_key_iv: "c24737c6bb507187ee24939e7c8cb6d5"
server_key_version: "encryptionkey@0"
I20260926 20:44:35.209400 9533 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260926 20:44:35.213030 9548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:35.214995 9533 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:44:35.215337 9533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "a00afe2786d449b4afa84acecb04227b"
format_stamp: "Formatted at 2026-09-26 20:44:35 on dist-test-slave-4d5t"
server_key: "c5b47b8ccf426d9a7badd472f673d7f6"
server_key_iv: "c24737c6bb507187ee24939e7c8cb6d5"
server_key_version: "encryptionkey@0"
I20260926 20:44:35.215611 9533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:35.242882 9533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:35.248647 9533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:35.249049 9533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:35.271085 9533 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:33235
I20260926 20:44:35.271119 9599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:33235 every 8 connection(s)
I20260926 20:44:35.272697 9533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:44:35.274819 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 9533
I20260926 20:44:35.275039 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:44:35.275699 23731 external_mini_cluster.cc:1428] Setting key ef9e51a6e56847b05187fe58dc59fddc
I20260926 20:44:35.278039 9600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:44:35 dist-test-slave-4d5t krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:44:35.295166 9600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b: Bootstrap starting.
I20260926 20:44:35.300848 9600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:35.302099 9600 log.cc:824] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:35.305370 9600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b: No bootstrap required, opened a new log
I20260926 20:44:35.305562 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:35.277751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53050 (local address 127.23.44.254:33235)
0926 20:44:35.278563 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:35.278575 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:35.278617 (+ 42us) server_negotiation.cc:407] Connection header received
0926 20:44:35.280314 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:35.280336 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:35.280813 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:35.281162 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:35.283394 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:35.284881 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:35.286933 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:35.287729 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:35.287896 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:35.294554 (+ 6658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:35.294573 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:35.294604 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:35.294679 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:35.299878 (+ 5199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:35.301275 (+ 1397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:35.301287 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:35.301297 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:35.301637 (+ 340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:35.302411 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:35.302419 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:35.302421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:35.303100 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:35.303454 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:35.303961 (+ 507us) server_negotiation.cc:299] Negotiation successful
0926 20:44:35.304436 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":376,"threads_started":1}
I20260926 20:44:35.315070 9600 raft_consensus.cc:348] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00afe2786d449b4afa84acecb04227b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33235 } }
I20260926 20:44:35.315533 9600 raft_consensus.cc:374] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:35.315668 9600 raft_consensus.cc:729] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a00afe2786d449b4afa84acecb04227b, State: Initialized, Role: FOLLOWER
I20260926 20:44:35.316401 9600 consensus_queue.cc:260] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00afe2786d449b4afa84acecb04227b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33235 } }
I20260926 20:44:35.316737 9600 raft_consensus.cc:388] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:44:35.316915 9600 raft_consensus.cc:482] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:44:35.317113 9600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:35.320087 9600 raft_consensus.cc:504] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00afe2786d449b4afa84acecb04227b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33235 } }
I20260926 20:44:35.320751 9600 leader_election.cc:302] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a00afe2786d449b4afa84acecb04227b; no voters:
I20260926 20:44:35.321370 9600 leader_election.cc:288] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:44:35.321599 9605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:35.322348 9605 raft_consensus.cc:686] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [term 1 LEADER]: Becoming Leader. State: Replica: a00afe2786d449b4afa84acecb04227b, State: Running, Role: LEADER
I20260926 20:44:35.322932 9605 consensus_queue.cc:237] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00afe2786d449b4afa84acecb04227b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33235 } }
I20260926 20:44:35.323599 9600 sys_catalog.cc:564] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:44:35.325862 9607 sys_catalog.cc:455] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a00afe2786d449b4afa84acecb04227b. Latest consensus state: current_term: 1 leader_uuid: "a00afe2786d449b4afa84acecb04227b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00afe2786d449b4afa84acecb04227b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33235 } } }
I20260926 20:44:35.326367 9607 sys_catalog.cc:458] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:35.326879 9606 sys_catalog.cc:455] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a00afe2786d449b4afa84acecb04227b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00afe2786d449b4afa84acecb04227b" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 33235 } } }
I20260926 20:44:35.327296 9606 sys_catalog.cc:458] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:35.327484 9612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:44:35.333380 9612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:44:35.342435 9612 catalog_manager.cc:1347] Generated new cluster ID: 5e5fed80efc24627a37d83e053e0fdba
I20260926 20:44:35.342566 9612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:44:35.361045 9612 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:44:35.363035 9612 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:44:35.372699 9612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b: Generated new TSK 0
I20260926 20:44:35.374204 9612 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:44:35.448891 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33235
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:35.802953 9626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:35.803309 9626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:35.803485 9626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:35.811419 9626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:35.811596 9626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:35.811785 9626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:44:35.822175 9626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:35.823643 9626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:35.825356 9626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:35.840816 9631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:35.840867 9632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:35.840890 9634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:35.841851 9626 server_base.cc:1034] running on GCE node
I20260926 20:44:35.842541 9626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:35.843580 9626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:35.844826 9626 hybrid_clock.cc:648] HybridClock initialized: now 1790455475844770 us; error 52 us; skew 500 ppm
I20260926 20:44:35.845194 9626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:35 dist-test-slave-4d5t krb5kdc[9520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455475, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:35.848750 9626 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:44:35.851411 9626 webserver.cc:460] Webserver started at http://127.23.44.193:35709/ using document root <none> and password file <none>
I20260926 20:44:35.852811 9626 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:35.853034 9626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:35.853407 9626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:35.856648 9626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "4546e4c4eee04635bd6245593ee662bc"
format_stamp: "Formatted at 2026-09-26 20:44:35 on dist-test-slave-4d5t"
server_key: "36d4f9b07419236a29a100458b6f4128"
server_key_iv: "83c720283fb1ecf0175a2c6f5bfd7950"
server_key_version: "encryptionkey@0"
I20260926 20:44:35.857491 9626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "4546e4c4eee04635bd6245593ee662bc"
format_stamp: "Formatted at 2026-09-26 20:44:35 on dist-test-slave-4d5t"
server_key: "36d4f9b07419236a29a100458b6f4128"
server_key_iv: "83c720283fb1ecf0175a2c6f5bfd7950"
server_key_version: "encryptionkey@0"
I20260926 20:44:35.863125 9626 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:44:35.866833 9641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:35.868250 9626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:44:35.868458 9626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "4546e4c4eee04635bd6245593ee662bc"
format_stamp: "Formatted at 2026-09-26 20:44:35 on dist-test-slave-4d5t"
server_key: "36d4f9b07419236a29a100458b6f4128"
server_key_iv: "83c720283fb1ecf0175a2c6f5bfd7950"
server_key_version: "encryptionkey@0"
I20260926 20:44:35.868749 9626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:35.892498 9626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:35.897873 9626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:35.898213 9626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:35.899276 9626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:35.900846 9626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:35.900950 9626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:35.901099 9626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:35.901172 9626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:35.941512 9626 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:35165
I20260926 20:44:35.941555 9753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:35165 every 8 connection(s)
I20260926 20:44:35.942998 9626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:44:35.951467 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 9626
I20260926 20:44:35.951682 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:44:35.952287 23731 external_mini_cluster.cc:1428] Setting key 1cfed39a5e330940038b2a6fa1456b02
Sep 26 20:44:35 dist-test-slave-4d5t krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455475, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:44:35.966612 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:35.945908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:37141 (local address 127.23.44.254:33235)
0926 20:44:35.946390 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:35.946395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:35.947545 (+ 1150us) server_negotiation.cc:407] Connection header received
0926 20:44:35.949867 (+ 2322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:35.949876 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:35.949977 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:35.950119 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:35.951956 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:35.952994 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:35.955099 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:35.955762 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:35.955908 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:35.959468 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:35.959478 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:35.959485 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:35.959533 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:35.963082 (+ 3549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:35.964035 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:35.964039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:35.964042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:35.964166 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:35.964722 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:35.964726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:35.964729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:35.965044 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:35.965171 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:35.966143 (+ 972us) server_negotiation.cc:299] Negotiation successful
0926 20:44:35.966342 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":223,"threads_started":1}
I20260926 20:44:35.968497 9757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:35.946521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33235 (local address 127.23.44.193:37141)
0926 20:44:35.947347 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:35.947389 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:35.949476 (+ 2087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:35.950391 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:35.950401 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:35.950994 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:35.951678 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:35.951704 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:35.953176 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:35.953187 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:35.954930 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:35.954942 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:35.956002 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:35.956008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:35.956208 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:35.957003 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:44:35.957031 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:35.959202 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:35.963246 (+ 4044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:35.963253 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:35.963268 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:35.963875 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:35.964289 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:35.964295 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:35.964299 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:35.964599 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:35.965183 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:35.965195 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:35.965694 (+ 499us) client_negotiation.cc:769] Sending connection context
0926 20:44:35.966001 (+ 307us) client_negotiation.cc:240] Negotiation successful
0926 20:44:35.966972 (+ 971us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":319,"threads_started":1}
I20260926 20:44:35.970319 9754 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33235
I20260926 20:44:35.970671 9754 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:35.971414 9754 heartbeater.cc:511] Master 127.23.44.254:33235 requested a full tablet report, sending...
I20260926 20:44:35.973757 9565 ts_manager.cc:194] Registered new tserver with Master: 4546e4c4eee04635bd6245593ee662bc (127.23.44.193:35165)
I20260926 20:44:35.975193 9565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:37141
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:44:36.017088 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33235
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:36.375015 9760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:36.375325 9760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:36.375500 9760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:36.383826 9760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:36.383952 9760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:36.384136 9760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:44:36.395237 9760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:36.396639 9760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:36.397970 9760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:36.411746 9765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:36.411710 9768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:36.411651 9766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:36.418006 9760 server_base.cc:1034] running on GCE node
I20260926 20:44:36.418685 9760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:36.419798 9760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:36.421038 9760 hybrid_clock.cc:648] HybridClock initialized: now 1790455476420983 us; error 47 us; skew 500 ppm
I20260926 20:44:36.421450 9760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:36 dist-test-slave-4d5t krb5kdc[9520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455476, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:36.424878 9760 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:44:36.426923 9760 webserver.cc:460] Webserver started at http://127.23.44.194:39871/ using document root <none> and password file <none>
I20260926 20:44:36.427699 9760 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:36.427832 9760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:36.428175 9760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:36.431393 9760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "3b8bf1d4b3224eb786bf927500ba9ee9"
format_stamp: "Formatted at 2026-09-26 20:44:36 on dist-test-slave-4d5t"
server_key: "786d7dc4a4a8b2ef58d2ae0b5e990aaf"
server_key_iv: "8ce52fd20a98a2d2f3bc20607c5567d6"
server_key_version: "encryptionkey@0"
I20260926 20:44:36.432220 9760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "3b8bf1d4b3224eb786bf927500ba9ee9"
format_stamp: "Formatted at 2026-09-26 20:44:36 on dist-test-slave-4d5t"
server_key: "786d7dc4a4a8b2ef58d2ae0b5e990aaf"
server_key_iv: "8ce52fd20a98a2d2f3bc20607c5567d6"
server_key_version: "encryptionkey@0"
I20260926 20:44:36.437943 9760 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:44:36.441877 9775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:36.443362 9760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:44:36.443513 9760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "3b8bf1d4b3224eb786bf927500ba9ee9"
format_stamp: "Formatted at 2026-09-26 20:44:36 on dist-test-slave-4d5t"
server_key: "786d7dc4a4a8b2ef58d2ae0b5e990aaf"
server_key_iv: "8ce52fd20a98a2d2f3bc20607c5567d6"
server_key_version: "encryptionkey@0"
I20260926 20:44:36.443797 9760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:36.477268 9760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:36.482076 9760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:36.482437 9760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:36.483546 9760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:36.485311 9760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:36.485409 9760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:36.485550 9760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:36.485636 9760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:36.526403 9760 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:43727
I20260926 20:44:36.526459 9887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:43727 every 8 connection(s)
I20260926 20:44:36.527891 9760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:44:36.537102 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 9760
I20260926 20:44:36.537312 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:44:36.537794 23731 external_mini_cluster.cc:1428] Setting key 524757ee8e8298c572f8842174b32085
Sep 26 20:44:36 dist-test-slave-4d5t krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455476, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:44:36.552511 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:36.530948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:39829 (local address 127.23.44.254:33235)
0926 20:44:36.531474 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:36.531480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:36.532394 (+ 914us) server_negotiation.cc:407] Connection header received
0926 20:44:36.534748 (+ 2354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:36.534754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:36.534843 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:36.534971 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:36.536694 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:36.537626 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:36.539972 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:36.540662 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:36.540806 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:36.544688 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:36.544694 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:36.544697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:36.544734 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:36.548018 (+ 3284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:36.549318 (+ 1300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:36.549323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:36.549325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:36.549464 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:36.550162 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:36.550168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:36.550171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:36.550667 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:36.550851 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:36.552009 (+ 1158us) server_negotiation.cc:299] Negotiation successful
0926 20:44:36.552219 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":277,"threads_started":1}
I20260926 20:44:36.554352 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:36.531407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33235 (local address 127.23.44.194:39829)
0926 20:44:36.532221 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:36.532270 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:36.534180 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:36.535277 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:36.535287 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:36.535843 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:36.536431 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:36.536450 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:36.537805 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:36.537810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:36.539798 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:36.539811 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:36.540910 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:36.540918 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:36.541164 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:36.541907 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:44:36.541930 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:36.544371 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:36.548214 (+ 3843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:36.548226 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:36.548250 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:36.549051 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:36.549617 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:36.549623 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:36.549628 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:36.549941 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:36.550890 (+ 949us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:36.550903 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:36.551496 (+ 593us) client_negotiation.cc:769] Sending connection context
0926 20:44:36.551861 (+ 365us) client_negotiation.cc:240] Negotiation successful
0926 20:44:36.552532 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":324,"threads_started":1}
I20260926 20:44:36.556033 9888 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33235
I20260926 20:44:36.556409 9888 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:36.557294 9888 heartbeater.cc:511] Master 127.23.44.254:33235 requested a full tablet report, sending...
I20260926 20:44:36.559044 9565 ts_manager.cc:194] Registered new tserver with Master: 3b8bf1d4b3224eb786bf927500ba9ee9 (127.23.44.194:43727)
I20260926 20:44:36.559993 9565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:39829
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:44:36.600831 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33235
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:39923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:36.963907 9894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:36.964255 9894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:36.964438 9894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:36.972895 9894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:36.973069 9894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:36.973268 9894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:44:36.978389 9754 heartbeater.cc:503] Master 127.23.44.254:33235 was elected leader, sending a full tablet report...
I20260926 20:44:36.986065 9894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:39923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:33235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:36.987514 9894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:36.988941 9894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:37.004305 9902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:37.004335 9899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:37.004360 9900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:37.005151 9894 server_base.cc:1034] running on GCE node
I20260926 20:44:37.005877 9894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:37.006973 9894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:37.008248 9894 hybrid_clock.cc:648] HybridClock initialized: now 1790455477008201 us; error 64 us; skew 500 ppm
I20260926 20:44:37.008697 9894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:37 dist-test-slave-4d5t krb5kdc[9520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455477, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:37.013156 9894 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:44:37.015259 9894 webserver.cc:460] Webserver started at http://127.23.44.195:45369/ using document root <none> and password file <none>
I20260926 20:44:37.016223 9894 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:37.016404 9894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:37.016867 9894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:37.020830 9894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "294efb8dcf8a4c22a38f498e26dd1e69"
format_stamp: "Formatted at 2026-09-26 20:44:37 on dist-test-slave-4d5t"
server_key: "fe42ab4fa939daf07431ac19a7ac0d33"
server_key_iv: "8434adedafb26f1dbc9eb8104459c788"
server_key_version: "encryptionkey@0"
I20260926 20:44:37.022222 9894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "294efb8dcf8a4c22a38f498e26dd1e69"
format_stamp: "Formatted at 2026-09-26 20:44:37 on dist-test-slave-4d5t"
server_key: "fe42ab4fa939daf07431ac19a7ac0d33"
server_key_iv: "8434adedafb26f1dbc9eb8104459c788"
server_key_version: "encryptionkey@0"
I20260926 20:44:37.028905 9894 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260926 20:44:37.032938 9909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:37.034441 9894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:44:37.034649 9894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "294efb8dcf8a4c22a38f498e26dd1e69"
format_stamp: "Formatted at 2026-09-26 20:44:37 on dist-test-slave-4d5t"
server_key: "fe42ab4fa939daf07431ac19a7ac0d33"
server_key_iv: "8434adedafb26f1dbc9eb8104459c788"
server_key_version: "encryptionkey@0"
I20260926 20:44:37.034893 9894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:37.062255 9894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:37.067281 9894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:37.067622 9894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:37.068848 9894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:37.070479 9894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:37.070598 9894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:37.070766 9894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:37.070855 9894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:37.108012 9894 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:40121
I20260926 20:44:37.108065 10021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:40121 every 8 connection(s)
I20260926 20:44:37.109401 9894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:44:37.112185 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 9894
I20260926 20:44:37.112396 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:44:37.112960 23731 external_mini_cluster.cc:1428] Setting key d46881658313f0da5e1b86338d862719
Sep 26 20:44:37 dist-test-slave-4d5t krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455477, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:44:37.135094 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.113541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:40111 (local address 127.23.44.254:33235)
0926 20:44:37.114223 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:37.114230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:37.115868 (+ 1638us) server_negotiation.cc:407] Connection header received
0926 20:44:37.118294 (+ 2426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:37.118302 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:37.118421 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:37.118563 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:37.120951 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.121788 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.123464 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.123942 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.124077 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.128319 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:37.128324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:37.128327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:37.128388 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:37.131997 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:37.132904 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:37.132908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:37.132911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:37.133031 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:37.133469 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:37.133472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:37.133474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:37.133741 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:37.133883 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:37.134604 (+ 721us) server_negotiation.cc:299] Negotiation successful
0926 20:44:37.134788 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":337,"threads_started":1}
I20260926 20:44:37.136039 10025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.114407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:33235 (local address 127.23.44.195:40111)
0926 20:44:37.115656 (+ 1249us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:37.115737 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:37.117966 (+ 2229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:37.118988 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:37.118999 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:37.119860 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:37.120764 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:37.120790 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.121945 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.121950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:37.123314 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:37.123326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.124176 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.124182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:37.124489 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.125520 (+ 1031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:44:37.125547 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:37.128106 (+ 2559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:37.132170 (+ 4064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:37.132178 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:37.132194 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:37.132782 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:37.133150 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:37.133153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:37.133156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:37.133353 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:37.133843 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:37.133849 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:37.134217 (+ 368us) client_negotiation.cc:769] Sending connection context
0926 20:44:37.134467 (+ 250us) client_negotiation.cc:240] Negotiation successful
0926 20:44:37.134888 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"thread_start_us":574,"threads_started":1}
I20260926 20:44:37.137805 10022 heartbeater.cc:347] Connected to a master server at 127.23.44.254:33235
I20260926 20:44:37.138098 10022 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:37.138888 10022 heartbeater.cc:511] Master 127.23.44.254:33235 requested a full tablet report, sending...
I20260926 20:44:37.140542 9565 ts_manager.cc:194] Registered new tserver with Master: 294efb8dcf8a4c22a38f498e26dd1e69 (127.23.44.195:40121)
I20260926 20:44:37.141657 9565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:40111
I20260926 20:44:37.143008 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:44:37.163442 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.147806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53876 (local address 127.23.44.254:33235)
0926 20:44:37.148066 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:37.148073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:37.148359 (+ 286us) server_negotiation.cc:407] Connection header received
0926 20:44:37.148636 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:37.148640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:37.148713 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:37.148795 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:37.150319 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.151045 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.152976 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.153397 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.153507 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.156667 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:37.156674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:37.156678 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:37.156716 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:37.159839 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:37.160995 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:37.161000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:37.161003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:37.161169 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:37.161705 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:37.161710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:37.161712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:37.162106 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:37.162366 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:37.162775 (+ 409us) server_negotiation.cc:299] Negotiation successful
0926 20:44:37.163025 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 20:44:37.168704 9565 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:44:37.176872 9565 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:44:37.220072 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.192439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47608 (local address 127.23.44.193:35165)
0926 20:44:37.210369 (+ 17930us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:37.210386 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:37.210437 (+ 51us) server_negotiation.cc:407] Connection header received
0926 20:44:37.210603 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:37.210628 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:37.211092 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:37.212717 (+ 1625us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:37.213027 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.214478 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.217358 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.218704 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.218925 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.219193 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:37.219523 (+ 330us) server_negotiation.cc:299] Negotiation successful
0926 20:44:37.219701 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17537,"thread_start_us":19963,"threads_started":1}
I20260926 20:44:37.220139 10040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.192033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:35165 (local address 127.0.0.1:47608)
0926 20:44:37.203848 (+ 11815us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:37.203934 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:37.204300 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:37.211530 (+ 7230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:37.211543 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:37.211591 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:37.212211 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:37.212224 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.214688 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.214695 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:37.216959 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:37.216977 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.218988 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.219000 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:37.219243 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.219298 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 20:44:37.219452 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 20:44:37.219678 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11304,"thread_start_us":15363,"threads_started":1}
I20260926 20:44:37.222384 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.193177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:40121 (local address 127.0.0.1:59688)
0926 20:44:37.209702 (+ 16525us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:37.209725 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:37.209835 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:37.213239 (+ 3404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:37.213245 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:37.213275 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:37.213669 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:37.213682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.215840 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.215846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:37.218699 (+ 2853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:37.218720 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.221399 (+ 2679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.221427 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:37.221610 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.221643 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 20:44:37.221746 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 20:44:37.221935 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16398,"mutex_wait_us":3,"thread_start_us":9859,"threads_started":1}
I20260926 20:44:37.222460 10037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.192313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59688 (local address 127.23.44.195:40121)
0926 20:44:37.212519 (+ 20206us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:37.212528 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:37.212589 (+ 61us) server_negotiation.cc:407] Connection header received
0926 20:44:37.212707 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:37.212718 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:37.212989 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:37.213228 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:37.213872 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.215598 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.219008 (+ 3410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.220805 (+ 1797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.220958 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.221215 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:37.221898 (+ 683us) server_negotiation.cc:299] Negotiation successful
0926 20:44:37.222094 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20000,"thread_start_us":19801,"threads_started":1}
I20260926 20:44:37.224619 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.192289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:43727 (local address 127.0.0.1:34844)
0926 20:44:37.207850 (+ 15561us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:37.207893 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:37.208132 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:37.214369 (+ 6237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:37.214383 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:37.214413 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:37.214921 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:37.214933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.217571 (+ 2638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.217589 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:37.220300 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:37.220317 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.223672 (+ 3355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.223679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:37.223895 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.223930 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 20:44:37.224062 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 20:44:37.224200 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15316,"mutex_wait_us":74,"thread_start_us":17123,"threads_started":1}
I20260926 20:44:37.224859 10039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.193130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34844 (local address 127.23.44.194:43727)
0926 20:44:37.213668 (+ 20538us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:37.213676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:37.213698 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:44:37.213820 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:37.213828 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:37.214040 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:37.214374 (+ 334us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:37.215133 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.217257 (+ 2124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.220633 (+ 3376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.223447 (+ 2814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.223617 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.223833 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:37.224123 (+ 290us) server_negotiation.cc:299] Negotiation successful
0926 20:44:37.224320 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20391,"thread_start_us":20005,"threads_started":1}
I20260926 20:44:37.224838 9689 tablet_service.cc:1467] Processing CreateTablet for tablet 143757c63d5c47329dbbcc857b7523b1 (DEFAULT_TABLE table=test-table [id=3a27821b161c47789c03846f6235c318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:37.226529 9689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 143757c63d5c47329dbbcc857b7523b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:37.226918 9957 tablet_service.cc:1467] Processing CreateTablet for tablet 143757c63d5c47329dbbcc857b7523b1 (DEFAULT_TABLE table=test-table [id=3a27821b161c47789c03846f6235c318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:37.228449 9957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 143757c63d5c47329dbbcc857b7523b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:37.228732 9823 tablet_service.cc:1467] Processing CreateTablet for tablet 143757c63d5c47329dbbcc857b7523b1 (DEFAULT_TABLE table=test-table [id=3a27821b161c47789c03846f6235c318]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:37.230090 9823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 143757c63d5c47329dbbcc857b7523b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:37.242703 10041 tablet_bootstrap.cc:492] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69: Bootstrap starting.
I20260926 20:44:37.244863 10042 tablet_bootstrap.cc:492] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc: Bootstrap starting.
I20260926 20:44:37.246302 10043 tablet_bootstrap.cc:492] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9: Bootstrap starting.
I20260926 20:44:37.247125 10041 tablet_bootstrap.cc:654] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:37.248492 10041 log.cc:824] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:37.248536 10042 tablet_bootstrap.cc:654] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:37.249420 10043 tablet_bootstrap.cc:654] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:37.250084 10042 log.cc:824] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:37.251386 10043 log.cc:824] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:37.253495 10042 tablet_bootstrap.cc:492] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc: No bootstrap required, opened a new log
I20260926 20:44:37.253794 10042 ts_tablet_manager.cc:1397] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260926 20:44:37.254122 10043 tablet_bootstrap.cc:492] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9: No bootstrap required, opened a new log
I20260926 20:44:37.254374 10043 ts_tablet_manager.cc:1397] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260926 20:44:37.254679 10041 tablet_bootstrap.cc:492] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69: No bootstrap required, opened a new log
I20260926 20:44:37.254930 10041 ts_tablet_manager.cc:1397] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260926 20:44:37.262040 10043 raft_consensus.cc:348] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4546e4c4eee04635bd6245593ee662bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35165 } } peers { permanent_uuid: "294efb8dcf8a4c22a38f498e26dd1e69" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40121 } } peers { permanent_uuid: "3b8bf1d4b3224eb786bf927500ba9ee9" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 43727 } }
I20260926 20:44:37.262454 10043 raft_consensus.cc:374] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:37.262549 10043 raft_consensus.cc:729] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8bf1d4b3224eb786bf927500ba9ee9, State: Initialized, Role: FOLLOWER
I20260926 20:44:37.263190 10043 consensus_queue.cc:260] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4546e4c4eee04635bd6245593ee662bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35165 } } peers { permanent_uuid: "294efb8dcf8a4c22a38f498e26dd1e69" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40121 } } peers { permanent_uuid: "3b8bf1d4b3224eb786bf927500ba9ee9" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 43727 } }
I20260926 20:44:37.264362 9888 heartbeater.cc:503] Master 127.23.44.254:33235 was elected leader, sending a full tablet report...
I20260926 20:44:37.264963 10043 ts_tablet_manager.cc:1428] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9: Time spent starting tablet: real 0.010s user 0.001s sys 0.011s
I20260926 20:44:37.264856 10042 raft_consensus.cc:348] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4546e4c4eee04635bd6245593ee662bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35165 } } peers { permanent_uuid: "294efb8dcf8a4c22a38f498e26dd1e69" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40121 } } peers { permanent_uuid: "3b8bf1d4b3224eb786bf927500ba9ee9" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 43727 } }
I20260926 20:44:37.265273 10042 raft_consensus.cc:374] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:37.265386 10042 raft_consensus.cc:729] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4546e4c4eee04635bd6245593ee662bc, State: Initialized, Role: FOLLOWER
I20260926 20:44:37.265992 10042 consensus_queue.cc:260] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4546e4c4eee04635bd6245593ee662bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35165 } } peers { permanent_uuid: "294efb8dcf8a4c22a38f498e26dd1e69" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40121 } } peers { permanent_uuid: "3b8bf1d4b3224eb786bf927500ba9ee9" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 43727 } }
I20260926 20:44:37.267359 10041 raft_consensus.cc:348] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4546e4c4eee04635bd6245593ee662bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35165 } } peers { permanent_uuid: "294efb8dcf8a4c22a38f498e26dd1e69" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40121 } } peers { permanent_uuid: "3b8bf1d4b3224eb786bf927500ba9ee9" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 43727 } }
I20260926 20:44:37.267748 10042 ts_tablet_manager.cc:1428] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260926 20:44:37.267807 10041 raft_consensus.cc:374] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:37.267894 10041 raft_consensus.cc:729] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 294efb8dcf8a4c22a38f498e26dd1e69, State: Initialized, Role: FOLLOWER
I20260926 20:44:37.268632 10041 consensus_queue.cc:260] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4546e4c4eee04635bd6245593ee662bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35165 } } peers { permanent_uuid: "294efb8dcf8a4c22a38f498e26dd1e69" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40121 } } peers { permanent_uuid: "3b8bf1d4b3224eb786bf927500ba9ee9" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 43727 } }
I20260926 20:44:37.269775 10022 heartbeater.cc:503] Master 127.23.44.254:33235 was elected leader, sending a full tablet report...
I20260926 20:44:37.276283 10041 ts_tablet_manager.cc:1428] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69: Time spent starting tablet: real 0.021s user 0.015s sys 0.000s
W20260926 20:44:37.280445 9889 tablet.cc:2369] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:44:37.361353 10023 tablet.cc:2369] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:44:37.446012 9755 tablet.cc:2369] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:44:37.460132 10048 raft_consensus.cc:482] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:44:37.460778 10048 raft_consensus.cc:504] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4546e4c4eee04635bd6245593ee662bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35165 } } peers { permanent_uuid: "294efb8dcf8a4c22a38f498e26dd1e69" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40121 } } peers { permanent_uuid: "3b8bf1d4b3224eb786bf927500ba9ee9" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 43727 } }
I20260926 20:44:37.463001 10048 leader_election.cc:288] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 294efb8dcf8a4c22a38f498e26dd1e69 (127.23.44.195:40121), 3b8bf1d4b3224eb786bf927500ba9ee9 (127.23.44.194:43727)
I20260926 20:44:37.471154 10037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.463462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:58099 (local address 127.23.44.195:40121)
0926 20:44:37.463695 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:37.463705 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:37.464704 (+ 999us) server_negotiation.cc:407] Connection header received
0926 20:44:37.464974 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:37.464979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:37.465058 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:37.465165 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:37.465796 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.466627 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.469025 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.469807 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.469973 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.470059 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:37.470736 (+ 677us) server_negotiation.cc:299] Negotiation successful
0926 20:44:37.470874 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 20:44:37.471418 10050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.463435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:43727 (local address 127.23.44.193:51293)
0926 20:44:37.464381 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:37.464398 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:37.464532 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:37.464990 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:37.464994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:37.465016 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:37.465434 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:37.465444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.466501 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.466506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:37.468335 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:37.468348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.470321 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.470326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:37.470559 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.470589 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 20:44:37.470748 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 20:44:37.470905 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":106,"thread_start_us":545,"threads_started":1}
I20260926 20:44:37.471341 10039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.463548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:51293 (local address 127.23.44.194:43727)
0926 20:44:37.463796 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:37.463805 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:37.464496 (+ 691us) server_negotiation.cc:407] Connection header received
0926 20:44:37.464733 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:37.464737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:37.464834 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:37.464936 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:37.465591 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.466364 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.468754 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.469829 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.470216 (+ 387us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.470313 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:37.470798 (+ 485us) server_negotiation.cc:299] Negotiation successful
0926 20:44:37.471015 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 20:44:37.471657 10051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.463587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:40121 (local address 127.23.44.193:58099)
0926 20:44:37.464540 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:37.464610 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:37.464829 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:37.465189 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:37.465192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:37.465216 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:37.465603 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:37.465615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.466771 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.466774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:37.468798 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:37.468826 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.470258 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.470265 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:37.470494 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.470555 (+ 61us) client_negotiation.cc:769] Sending connection context
0926 20:44:37.470657 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 20:44:37.470848 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":6,"thread_start_us":418,"threads_started":1}
I20260926 20:44:37.472534 9843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "143757c63d5c47329dbbcc857b7523b1" candidate_uuid: "4546e4c4eee04635bd6245593ee662bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8bf1d4b3224eb786bf927500ba9ee9" is_pre_election: true
I20260926 20:44:37.472782 9977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "143757c63d5c47329dbbcc857b7523b1" candidate_uuid: "4546e4c4eee04635bd6245593ee662bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294efb8dcf8a4c22a38f498e26dd1e69" is_pre_election: true
I20260926 20:44:37.473239 9843 raft_consensus.cc:2454] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4546e4c4eee04635bd6245593ee662bc in term 0.
I20260926 20:44:37.473309 9977 raft_consensus.cc:2454] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4546e4c4eee04635bd6245593ee662bc in term 0.
I20260926 20:44:37.474028 9642 leader_election.cc:302] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b8bf1d4b3224eb786bf927500ba9ee9, 4546e4c4eee04635bd6245593ee662bc; no voters:
I20260926 20:44:37.474472 10048 raft_consensus.cc:2781] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:44:37.474638 10048 raft_consensus.cc:482] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:44:37.474786 10048 raft_consensus.cc:3037] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:37.477322 10048 raft_consensus.cc:504] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4546e4c4eee04635bd6245593ee662bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35165 } } peers { permanent_uuid: "294efb8dcf8a4c22a38f498e26dd1e69" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40121 } } peers { permanent_uuid: "3b8bf1d4b3224eb786bf927500ba9ee9" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 43727 } }
I20260926 20:44:37.478080 10048 leader_election.cc:288] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [CANDIDATE]: Term 1 election: Requested vote from peers 294efb8dcf8a4c22a38f498e26dd1e69 (127.23.44.195:40121), 3b8bf1d4b3224eb786bf927500ba9ee9 (127.23.44.194:43727)
I20260926 20:44:37.478617 9977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "143757c63d5c47329dbbcc857b7523b1" candidate_uuid: "4546e4c4eee04635bd6245593ee662bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "294efb8dcf8a4c22a38f498e26dd1e69"
I20260926 20:44:37.478713 9843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "143757c63d5c47329dbbcc857b7523b1" candidate_uuid: "4546e4c4eee04635bd6245593ee662bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b8bf1d4b3224eb786bf927500ba9ee9"
I20260926 20:44:37.478842 9977 raft_consensus.cc:3037] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:37.478996 9843 raft_consensus.cc:3037] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:37.481374 9843 raft_consensus.cc:2454] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4546e4c4eee04635bd6245593ee662bc in term 1.
I20260926 20:44:37.481942 9977 raft_consensus.cc:2454] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4546e4c4eee04635bd6245593ee662bc in term 1.
I20260926 20:44:37.482019 9642 leader_election.cc:302] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b8bf1d4b3224eb786bf927500ba9ee9, 4546e4c4eee04635bd6245593ee662bc; no voters:
I20260926 20:44:37.482555 10048 raft_consensus.cc:2781] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:37.483114 10048 raft_consensus.cc:686] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [term 1 LEADER]: Becoming Leader. State: Replica: 4546e4c4eee04635bd6245593ee662bc, State: Running, Role: LEADER
I20260926 20:44:37.483611 10048 consensus_queue.cc:237] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4546e4c4eee04635bd6245593ee662bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35165 } } peers { permanent_uuid: "294efb8dcf8a4c22a38f498e26dd1e69" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40121 } } peers { permanent_uuid: "3b8bf1d4b3224eb786bf927500ba9ee9" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 43727 } }
I20260926 20:44:37.488903 9565 catalog_manager.cc:5495] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 4546e4c4eee04635bd6245593ee662bc (127.23.44.193). New cstate: current_term: 1 leader_uuid: "4546e4c4eee04635bd6245593ee662bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4546e4c4eee04635bd6245593ee662bc" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 35165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "294efb8dcf8a4c22a38f498e26dd1e69" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b8bf1d4b3224eb786bf927500ba9ee9" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 43727 } health_report { overall_health: UNKNOWN } } }
I20260926 20:44:37.520891 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.515430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47616 (local address 127.23.44.193:35165)
0926 20:44:37.515613 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:37.515618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:37.515634 (+ 16us) server_negotiation.cc:407] Connection header received
0926 20:44:37.515722 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:37.515726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:37.515802 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:37.515927 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:44:37.516305 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.517104 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.518536 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.518963 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.519070 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.519637 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:44:37.520177 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:44:37.520312 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:37.520533 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 20:44:37.520631 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 20:44:37.531332 9977 raft_consensus.cc:1261] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69 [term 1 FOLLOWER]: Refusing update from remote peer 4546e4c4eee04635bd6245593ee662bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:44:37.531538 9843 raft_consensus.cc:1261] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9 [term 1 FOLLOWER]: Refusing update from remote peer 4546e4c4eee04635bd6245593ee662bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:44:37.532582 10048 consensus_queue.cc:1035] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "294efb8dcf8a4c22a38f498e26dd1e69" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:44:37.532997 10052 consensus_queue.cc:1035] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b8bf1d4b3224eb786bf927500ba9ee9" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 43727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:44:37.548588 10057 mvcc.cc:205] Tried to move back new op lower bound from 7333705635954774016 to 7333705635776524288. Current Snapshot: MvccSnapshot[applied={T|T < 7333705635954774016}]
I20260926 20:44:37.556618 10056 mvcc.cc:205] Tried to move back new op lower bound from 7333705635954774016 to 7333705635776524288. Current Snapshot: MvccSnapshot[applied={T|T < 7333705635954774016}]
I20260926 20:44:37.569070 9565 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:44:37.569370 9565 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:44:37.573377 9565 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a00afe2786d449b4afa84acecb04227b: Sending DeleteTablet for 3 replicas of tablet 143757c63d5c47329dbbcc857b7523b1
I20260926 20:44:37.574716 9957 tablet_service.cc:1514] Processing DeleteTablet for tablet 143757c63d5c47329dbbcc857b7523b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:37 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:59688
I20260926 20:44:37.574790 9823 tablet_service.cc:1514] Processing DeleteTablet for tablet 143757c63d5c47329dbbcc857b7523b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:37 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:34844
I20260926 20:44:37.575131 9689 tablet_service.cc:1514] Processing DeleteTablet for tablet 143757c63d5c47329dbbcc857b7523b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:37 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:47608
I20260926 20:44:37.577632 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 9626
I20260926 20:44:37.588203 10065 tablet_replica.cc:330] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9: stopping tablet replica
I20260926 20:44:37.588958 10065 raft_consensus.cc:2229] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:44:37.589696 10065 raft_consensus.cc:2258] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:37.591506 10066 tablet_replica.cc:330] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc: stopping tablet replica
I20260926 20:44:37.592094 10066 raft_consensus.cc:2229] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:44:37.593056 10066 raft_consensus.cc:2258] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:37.595726 10066 ts_tablet_manager.cc:1905] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:37.596422 10064 tablet_replica.cc:330] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69: stopping tablet replica
I20260926 20:44:37.596958 10064 raft_consensus.cc:2229] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:44:37.597522 10065 ts_tablet_manager.cc:1905] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:37.598126 10064 raft_consensus.cc:2258] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69 [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 26 20:44:37 dist-test-slave-4d5t krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:44:37.606865 10064 ts_tablet_manager.cc:1905] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:37.607555 10065 ts_tablet_manager.cc:1918] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:37.607779 10065 log.cc:1196] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/143757c63d5c47329dbbcc857b7523b1
I20260926 20:44:37.608982 10065 ts_tablet_manager.cc:1939] T 143757c63d5c47329dbbcc857b7523b1 P 3b8bf1d4b3224eb786bf927500ba9ee9: Deleting consensus metadata
I20260926 20:44:37.611153 9549 catalog_manager.cc:4841] TS 3b8bf1d4b3224eb786bf927500ba9ee9 (127.23.44.194:43727): tablet 143757c63d5c47329dbbcc857b7523b1 (table test-table [id=3a27821b161c47789c03846f6235c318]) successfully deleted
I20260926 20:44:37.614219 10066 ts_tablet_manager.cc:1918] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:37.614207 10064 ts_tablet_manager.cc:1918] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:37.614405 10064 log.cc:1196] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/143757c63d5c47329dbbcc857b7523b1
I20260926 20:44:37.614413 10066 log.cc:1196] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/143757c63d5c47329dbbcc857b7523b1
I20260926 20:44:37.614943 10064 ts_tablet_manager.cc:1939] T 143757c63d5c47329dbbcc857b7523b1 P 294efb8dcf8a4c22a38f498e26dd1e69: Deleting consensus metadata
I20260926 20:44:37.615007 10066 ts_tablet_manager.cc:1939] T 143757c63d5c47329dbbcc857b7523b1 P 4546e4c4eee04635bd6245593ee662bc: Deleting consensus metadata
I20260926 20:44:37.615695 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.579383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47622 (local address 127.23.44.193:35165)
0926 20:44:37.579618 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:37.579627 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:37.590755 (+ 11128us) server_negotiation.cc:407] Connection header received
0926 20:44:37.591430 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:37.591444 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:37.591581 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:37.591725 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:37.594250 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.595803 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.597808 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.598532 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.598641 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.605815 (+ 7174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:37.605834 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:37.605858 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:37.605918 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:37.611403 (+ 5485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:37.612471 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:37.612482 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:37.612488 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:37.612738 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:37.613205 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:37.613211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:37.613214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:37.613701 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:37.614038 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:37.614902 (+ 864us) server_negotiation.cc:299] Negotiation successful
0926 20:44:37.615309 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:44:37.616982 9551 catalog_manager.cc:4841] TS 294efb8dcf8a4c22a38f498e26dd1e69 (127.23.44.195:40121): tablet 143757c63d5c47329dbbcc857b7523b1 (table test-table [id=3a27821b161c47789c03846f6235c318]) successfully deleted
I20260926 20:44:37.617105 9550 catalog_manager.cc:4841] TS 4546e4c4eee04635bd6245593ee662bc (127.23.44.193:35165): tablet 143757c63d5c47329dbbcc857b7523b1 (table test-table [id=3a27821b161c47789c03846f6235c318]) successfully deleted
I20260926 20:44:37.617563 9749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:37.719920 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 9626
I20260926 20:44:37.738458 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 9760
Sep 26 20:44:37 dist-test-slave-4d5t krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:44:37.755784 10039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.739457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34848 (local address 127.23.44.194:43727)
0926 20:44:37.739641 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:37.739646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:37.739665 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:44:37.739725 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:37.739729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:37.739828 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:37.739915 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:37.741399 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.742283 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.743725 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.744162 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.744274 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.749441 (+ 5167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:37.749458 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:37.749475 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:37.749518 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:37.752859 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:37.753664 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:37.753672 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:37.753674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:37.753854 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:37.754285 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:37.754289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:37.754291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:37.754718 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:37.754949 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:37.755317 (+ 368us) server_negotiation.cc:299] Negotiation successful
0926 20:44:37.755513 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 20:44:37.756810 9883 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:37.852635 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 9760
I20260926 20:44:37.870586 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 9894
Sep 26 20:44:37 dist-test-slave-4d5t krb5kdc[9520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:44:37.886801 10037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:37.871441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59692 (local address 127.23.44.195:40121)
0926 20:44:37.871607 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:37.871611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:37.871628 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:44:37.871740 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:37.871744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:37.871808 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:37.871899 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:37.873488 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.874216 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.875594 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:37.876132 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:37.876252 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:37.880744 (+ 4492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:37.880759 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:37.880777 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:37.880820 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:37.883418 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:37.884164 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:37.884171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:37.884174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:37.884300 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:37.885196 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:37.885200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:37.885202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:37.885621 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:37.885877 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:37.886295 (+ 418us) server_negotiation.cc:299] Negotiation successful
0926 20:44:37.886465 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 20:44:37.887470 10017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:37.981189 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 9894
I20260926 20:44:37.998309 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 9533
I20260926 20:44:37.999703 9595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:38.114542 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 9533
2026-09-26T20:44:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4764 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T20:44:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:44:38Z Disabled control of system clock
I20260926 20:44:38.197296 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:45905
--webserver_interface=127.23.44.254
--webserver_port=0
--builtin_ntp_servers=127.23.44.212:42353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:45905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 20:44:38.546401 10075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:38.546717 10075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:38.546801 10075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:38.554208 10075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:44:38.554325 10075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:38.554364 10075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:38.554415 10075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:44:38.554461 10075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:44:38.564014 10075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:42353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:45905
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:45905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:38.565541 10075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:38.567412 10075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:38.579789 10080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:38.579963 10081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:38.580442 10083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:38.580955 10075 server_base.cc:1034] running on GCE node
I20260926 20:44:38.581887 10075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:38.583488 10075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:38.584745 10075 hybrid_clock.cc:648] HybridClock initialized: now 1790455478584707 us; error 45 us; skew 500 ppm
I20260926 20:44:38.585136 10075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:44:38.587553 10075 webserver.cc:460] Webserver started at http://127.23.44.254:37107/ using document root <none> and password file <none>
I20260926 20:44:38.588299 10075 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:38.588392 10075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:38.588737 10075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:38.592391 10075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "b37cc7b2ada94c0b8637d9fb08f01a60"
format_stamp: "Formatted at 2026-09-26 20:44:38 on dist-test-slave-4d5t"
server_key: "8309415708915e18a1a335464c10adfb"
server_key_iv: "1dd39d02349303c6fad52931bc5c14fb"
server_key_version: "encryptionkey@0"
I20260926 20:44:38.593434 10075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "b37cc7b2ada94c0b8637d9fb08f01a60"
format_stamp: "Formatted at 2026-09-26 20:44:38 on dist-test-slave-4d5t"
server_key: "8309415708915e18a1a335464c10adfb"
server_key_iv: "1dd39d02349303c6fad52931bc5c14fb"
server_key_version: "encryptionkey@0"
I20260926 20:44:38.599370 10075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:44:38.603253 10089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:38.605013 10075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 20:44:38.605228 10075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "b37cc7b2ada94c0b8637d9fb08f01a60"
format_stamp: "Formatted at 2026-09-26 20:44:38 on dist-test-slave-4d5t"
server_key: "8309415708915e18a1a335464c10adfb"
server_key_iv: "1dd39d02349303c6fad52931bc5c14fb"
server_key_version: "encryptionkey@0"
I20260926 20:44:38.605481 10075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:38.623709 10075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:38.624774 10075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:38.625103 10075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:38.647495 10075 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:45905
I20260926 20:44:38.647495 10140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:45905 every 8 connection(s)
I20260926 20:44:38.649412 10075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:44:38.654199 10141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:38.655468 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 10075
I20260926 20:44:38.655650 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:44:38.656172 23731 external_mini_cluster.cc:1428] Setting key a9236b7d22bb74328b891f6c663a87d1
I20260926 20:44:38.667789 10141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60: Bootstrap starting.
I20260926 20:44:38.671252 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:38.658238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47872 (local address 127.23.44.254:45905)
0926 20:44:38.659399 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:38.659415 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:38.659458 (+ 43us) server_negotiation.cc:407] Connection header received
0926 20:44:38.661285 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:38.661311 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:38.661846 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:38.662274 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:38.662860 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:38.664111 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:38.665621 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:38.666353 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:38.666546 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:38.668207 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:38.668217 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:38.668236 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:38.668991 (+ 755us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:38.669104 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:38.669134 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:38.669275 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:38.669570 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 20:44:38.669972 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":450,"threads_started":1}
I20260926 20:44:38.672057 10141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:38.673247 10141 log.cc:824] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:38.676379 10141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60: No bootstrap required, opened a new log
I20260926 20:44:38.684959 10141 raft_consensus.cc:348] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37cc7b2ada94c0b8637d9fb08f01a60" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45905 } }
I20260926 20:44:38.685516 10141 raft_consensus.cc:374] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:38.685645 10141 raft_consensus.cc:729] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37cc7b2ada94c0b8637d9fb08f01a60, State: Initialized, Role: FOLLOWER
I20260926 20:44:38.686314 10141 consensus_queue.cc:260] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37cc7b2ada94c0b8637d9fb08f01a60" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45905 } }
I20260926 20:44:38.686595 10141 raft_consensus.cc:388] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:44:38.686757 10141 raft_consensus.cc:482] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:44:38.686960 10141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:38.689611 10141 raft_consensus.cc:504] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37cc7b2ada94c0b8637d9fb08f01a60" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45905 } }
I20260926 20:44:38.690176 10141 leader_election.cc:302] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b37cc7b2ada94c0b8637d9fb08f01a60; no voters:
I20260926 20:44:38.690940 10141 leader_election.cc:288] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:44:38.691365 10146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:38.692240 10146 raft_consensus.cc:686] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [term 1 LEADER]: Becoming Leader. State: Replica: b37cc7b2ada94c0b8637d9fb08f01a60, State: Running, Role: LEADER
I20260926 20:44:38.692888 10146 consensus_queue.cc:237] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37cc7b2ada94c0b8637d9fb08f01a60" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45905 } }
I20260926 20:44:38.693516 10141 sys_catalog.cc:564] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:44:38.695776 10147 sys_catalog.cc:455] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b37cc7b2ada94c0b8637d9fb08f01a60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37cc7b2ada94c0b8637d9fb08f01a60" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45905 } } }
I20260926 20:44:38.696389 10147 sys_catalog.cc:458] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:38.696039 10148 sys_catalog.cc:455] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b37cc7b2ada94c0b8637d9fb08f01a60. Latest consensus state: current_term: 1 leader_uuid: "b37cc7b2ada94c0b8637d9fb08f01a60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37cc7b2ada94c0b8637d9fb08f01a60" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 45905 } } }
I20260926 20:44:38.696736 10148 sys_catalog.cc:458] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60 [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:38.698431 10153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:44:38.704325 10153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:44:38.713887 10153 catalog_manager.cc:1347] Generated new cluster ID: 72a0a0005536497899fb79c52cbc3265
I20260926 20:44:38.714023 10153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:44:38.743273 10153 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:44:38.745196 10153 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:44:38.757085 10153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60: Generated new TSK 0
I20260926 20:44:38.758239 10153 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:44:38.773239 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45905
--builtin_ntp_servers=127.23.44.212:42353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 20:44:39.126549 10165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:39.126875 10165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:39.126968 10165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:39.134644 10165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:39.134781 10165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:39.134945 10165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:44:39.144582 10165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:42353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:39.145946 10165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:39.147781 10165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:39.162499 10170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:39.162585 10173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:39.162401 10171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:39.163849 10165 server_base.cc:1034] running on GCE node
I20260926 20:44:39.164525 10165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:39.165817 10165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:39.167150 10165 hybrid_clock.cc:648] HybridClock initialized: now 1790455479167064 us; error 80 us; skew 500 ppm
I20260926 20:44:39.167536 10165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:44:39.170248 10165 webserver.cc:460] Webserver started at http://127.23.44.193:35825/ using document root <none> and password file <none>
I20260926 20:44:39.170984 10165 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:39.171120 10165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:39.171453 10165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:39.174530 10165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "9767880fc61a482e89019fc59e7d4776"
format_stamp: "Formatted at 2026-09-26 20:44:39 on dist-test-slave-4d5t"
server_key: "19f6bbc251d3207024c5bdc66b048f5b"
server_key_iv: "8963605180e7ab31f7332fa77fac5139"
server_key_version: "encryptionkey@0"
I20260926 20:44:39.175388 10165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "9767880fc61a482e89019fc59e7d4776"
format_stamp: "Formatted at 2026-09-26 20:44:39 on dist-test-slave-4d5t"
server_key: "19f6bbc251d3207024c5bdc66b048f5b"
server_key_iv: "8963605180e7ab31f7332fa77fac5139"
server_key_version: "encryptionkey@0"
I20260926 20:44:39.180972 10165 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 20:44:39.184926 10179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:39.186429 10165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:44:39.186728 10165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "9767880fc61a482e89019fc59e7d4776"
format_stamp: "Formatted at 2026-09-26 20:44:39 on dist-test-slave-4d5t"
server_key: "19f6bbc251d3207024c5bdc66b048f5b"
server_key_iv: "8963605180e7ab31f7332fa77fac5139"
server_key_version: "encryptionkey@0"
I20260926 20:44:39.186978 10165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:39.218191 10165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:39.219079 10165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:39.219414 10165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:39.220510 10165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:39.222126 10165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:39.222258 10165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:39.222393 10165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:39.222482 10165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:39.259078 10165 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:44451
I20260926 20:44:39.259171 10291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:44451 every 8 connection(s)
I20260926 20:44:39.260437 10165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:44:39.262264 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 10165
I20260926 20:44:39.262465 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:44:39.263011 23731 external_mini_cluster.cc:1428] Setting key 33dc91e87bf90a5a0eef97ec412ea571
I20260926 20:44:39.266244 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45905
--builtin_ntp_servers=127.23.44.212:42353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 20:44:39.274605 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:39.263964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:48337 (local address 127.23.44.254:45905)
0926 20:44:39.264486 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:39.264493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:39.265863 (+ 1370us) server_negotiation.cc:407] Connection header received
0926 20:44:39.267842 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:39.267848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:39.267957 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:39.268082 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:39.269272 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:39.270000 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:39.271593 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:39.272009 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:39.272152 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:39.273716 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:39.273723 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:39.273732 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:39.273757 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:39.273815 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:39.273834 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:39.273943 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:39.274228 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 20:44:39.274339 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":266,"threads_started":1}
I20260926 20:44:39.275657 10295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:39.264665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45905 (local address 127.23.44.193:48337)
0926 20:44:39.265702 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:39.265746 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:39.267544 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:39.268321 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:39.268329 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:39.268422 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:39.269100 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:39.269116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:39.270172 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:39.270177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:39.271445 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:39.271457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:39.272250 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:39.272258 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:39.272497 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:39.273355 (+ 858us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:44:39.273358 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:39.273401 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:44:39.273491 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:39.273985 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:39.273994 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:39.274001 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 20:44:39.274161 (+ 160us) client_negotiation.cc:240] Negotiation successful
0926 20:44:39.274497 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":367,"threads_started":1}
I20260926 20:44:39.277444 10292 heartbeater.cc:347] Connected to a master server at 127.23.44.254:45905
I20260926 20:44:39.277809 10292 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:39.278787 10292 heartbeater.cc:511] Master 127.23.44.254:45905 requested a full tablet report, sending...
I20260926 20:44:39.281231 10106 ts_manager.cc:194] Registered new tserver with Master: 9767880fc61a482e89019fc59e7d4776 (127.23.44.193:44451)
I20260926 20:44:39.282737 10106 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.44.193:48337
W20260926 20:44:39.621438 10296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:39.621915 10296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:39.622112 10296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:39.630620 10296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:39.630738 10296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:39.630914 10296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:44:39.642268 10296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:42353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:39.643682 10296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:39.645666 10296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:39.661058 10302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:39.661123 10301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:39.661980 10304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:39.666951 10296 server_base.cc:1034] running on GCE node
I20260926 20:44:39.667728 10296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:39.668906 10296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:39.670312 10296 hybrid_clock.cc:648] HybridClock initialized: now 1790455479670243 us; error 76 us; skew 500 ppm
I20260926 20:44:39.670718 10296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:44:39.673614 10296 webserver.cc:460] Webserver started at http://127.23.44.194:35949/ using document root <none> and password file <none>
I20260926 20:44:39.674396 10296 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:39.674537 10296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:39.674866 10296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:39.678086 10296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "d0461bdf55e3470db62ce437a6c55f85"
format_stamp: "Formatted at 2026-09-26 20:44:39 on dist-test-slave-4d5t"
server_key: "8290d587bf61397dab7da1c1a8814cbd"
server_key_iv: "8ef9c559ea529d1a68b695be1062fa24"
server_key_version: "encryptionkey@0"
I20260926 20:44:39.678956 10296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "d0461bdf55e3470db62ce437a6c55f85"
format_stamp: "Formatted at 2026-09-26 20:44:39 on dist-test-slave-4d5t"
server_key: "8290d587bf61397dab7da1c1a8814cbd"
server_key_iv: "8ef9c559ea529d1a68b695be1062fa24"
server_key_version: "encryptionkey@0"
I20260926 20:44:39.684928 10296 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 20:44:39.688970 10310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:39.690681 10296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:44:39.690873 10296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "d0461bdf55e3470db62ce437a6c55f85"
format_stamp: "Formatted at 2026-09-26 20:44:39 on dist-test-slave-4d5t"
server_key: "8290d587bf61397dab7da1c1a8814cbd"
server_key_iv: "8ef9c559ea529d1a68b695be1062fa24"
server_key_version: "encryptionkey@0"
I20260926 20:44:39.691113 10296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:39.705081 10296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:39.705968 10296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:39.706277 10296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:39.707325 10296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:39.708958 10296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:39.709100 10296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:39.709229 10296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:39.709318 10296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:39.744820 10296 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:35849
I20260926 20:44:39.744858 10422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:35849 every 8 connection(s)
I20260926 20:44:39.746181 10296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:44:39.746675 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 10296
I20260926 20:44:39.746863 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:44:39.747462 23731 external_mini_cluster.cc:1428] Setting key a8baffad954b135781578beb82ab6697
I20260926 20:44:39.750765 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45905
--builtin_ntp_servers=127.23.44.212:42353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 20:44:39.763506 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:39.748882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:49495 (local address 127.23.44.254:45905)
0926 20:44:39.749117 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:39.749124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:39.750821 (+ 1697us) server_negotiation.cc:407] Connection header received
0926 20:44:39.753826 (+ 3005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:39.753831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:39.753907 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:39.754018 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:39.755683 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:39.756683 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:39.759549 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:39.760278 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:39.760441 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:39.762364 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:39.762371 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:39.762375 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:39.762405 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:39.762458 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:39.762475 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:39.762599 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:39.762961 (+ 362us) server_negotiation.cc:299] Negotiation successful
0926 20:44:39.763096 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:44:39.764894 10425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:39.749524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45905 (local address 127.23.44.194:49495)
0926 20:44:39.750617 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:39.750674 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:39.753448 (+ 2774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:39.754296 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:39.754310 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:39.754455 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:39.755350 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:39.755378 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:39.756903 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:39.756912 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:39.759307 (+ 2395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:39.759328 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:39.760605 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:39.760615 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:39.760977 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:39.761832 (+ 855us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:44:39.761838 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:39.761902 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:44:39.762017 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:39.762623 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:39.762637 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:39.762649 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 20:44:39.762879 (+ 230us) client_negotiation.cc:240] Negotiation successful
0926 20:44:39.763368 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":310,"threads_started":1}
I20260926 20:44:39.766460 10423 heartbeater.cc:347] Connected to a master server at 127.23.44.254:45905
I20260926 20:44:39.766798 10423 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:39.767719 10423 heartbeater.cc:511] Master 127.23.44.254:45905 requested a full tablet report, sending...
I20260926 20:44:39.769436 10106 ts_manager.cc:194] Registered new tserver with Master: d0461bdf55e3470db62ce437a6c55f85 (127.23.44.194:35849)
I20260926 20:44:39.770258 10106 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.44.194:49495
W20260926 20:44:40.104187 10426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:40.104527 10426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:40.104712 10426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:40.112470 10426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:40.112651 10426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:40.112828 10426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:44:40.122607 10426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:42353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:45905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:40.123955 10426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:40.125825 10426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:40.139740 10432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:40.139763 10431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:40.139771 10434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:40.140645 10426 server_base.cc:1034] running on GCE node
I20260926 20:44:40.141295 10426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:40.142414 10426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:40.143667 10426 hybrid_clock.cc:648] HybridClock initialized: now 1790455480143604 us; error 55 us; skew 500 ppm
I20260926 20:44:40.144030 10426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:44:40.151515 10426 webserver.cc:460] Webserver started at http://127.23.44.195:35159/ using document root <none> and password file <none>
I20260926 20:44:40.152359 10426 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:40.152455 10426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:40.152824 10426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:40.155884 10426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9"
format_stamp: "Formatted at 2026-09-26 20:44:40 on dist-test-slave-4d5t"
server_key: "e5903a23f6830b785ef076fd447274d5"
server_key_iv: "37253bd3073501ce47166cb70283d19a"
server_key_version: "encryptionkey@0"
I20260926 20:44:40.156787 10426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9"
format_stamp: "Formatted at 2026-09-26 20:44:40 on dist-test-slave-4d5t"
server_key: "e5903a23f6830b785ef076fd447274d5"
server_key_iv: "37253bd3073501ce47166cb70283d19a"
server_key_version: "encryptionkey@0"
I20260926 20:44:40.162261 10426 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 20:44:40.165802 10440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:40.167140 10426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:44:40.167328 10426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9"
format_stamp: "Formatted at 2026-09-26 20:44:40 on dist-test-slave-4d5t"
server_key: "e5903a23f6830b785ef076fd447274d5"
server_key_iv: "37253bd3073501ce47166cb70283d19a"
server_key_version: "encryptionkey@0"
I20260926 20:44:40.167568 10426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:40.192597 10426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:40.193550 10426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:40.193873 10426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:40.195012 10426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:40.196813 10426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:40.196939 10426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:40.197062 10426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:40.197148 10426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:40.236320 10426 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:41971
I20260926 20:44:40.236414 10552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:41971 every 8 connection(s)
I20260926 20:44:40.237736 10426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:44:40.244027 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 10426
I20260926 20:44:40.244385 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:44:40.245002 23731 external_mini_cluster.cc:1428] Setting key cfba1009dca9215274da5cd76e585eff
I20260926 20:44:40.252157 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.240510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:37697 (local address 127.23.44.254:45905)
0926 20:44:40.240747 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:40.240753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:40.242114 (+ 1361us) server_negotiation.cc:407] Connection header received
0926 20:44:40.244009 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:40.244015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:40.244109 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:40.244222 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:40.245492 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.246507 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.248354 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.248856 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.249123 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.250946 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:40.250951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:40.250954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:40.250974 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:40.251012 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:40.251025 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:40.251130 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:40.251370 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 20:44:40.251479 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 20:44:40.253748 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.240958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:45905 (local address 127.23.44.195:37697)
0926 20:44:40.241922 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:40.241967 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:40.243725 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:40.244469 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:40.244505 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:40.244650 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:40.245313 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:40.245332 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.246670 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.246674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:40.248187 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:40.248200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.249280 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.249297 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:40.249772 (+ 475us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.250571 (+ 799us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:44:40.250574 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:40.250617 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:44:40.250709 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:40.251128 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:40.251140 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:40.251151 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 20:44:40.251346 (+ 195us) client_negotiation.cc:240] Negotiation successful
0926 20:44:40.252164 (+ 818us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":304,"threads_started":1}
I20260926 20:44:40.255857 10553 heartbeater.cc:347] Connected to a master server at 127.23.44.254:45905
I20260926 20:44:40.256206 10553 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:40.257067 10553 heartbeater.cc:511] Master 127.23.44.254:45905 requested a full tablet report, sending...
I20260926 20:44:40.259753 10106 ts_manager.cc:194] Registered new tserver with Master: 595fbd8b3ed54c028e2cff30bbc5c4e9 (127.23.44.195:41971)
I20260926 20:44:40.261227 10106 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.44.195:37697
I20260926 20:44:40.262158 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:44:40.275449 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.267185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47874 (local address 127.23.44.254:45905)
0926 20:44:40.267365 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:40.267370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:40.268270 (+ 900us) server_negotiation.cc:407] Connection header received
0926 20:44:40.268443 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:40.268448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:40.268514 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:40.268687 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:40.269271 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.269984 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.271741 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.272541 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.272820 (+ 279us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.274200 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:40.274211 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:40.274215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:40.274247 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:40.274288 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:40.274302 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:40.274508 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:40.274951 (+ 443us) server_negotiation.cc:299] Negotiation successful
0926 20:44:40.275041 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 20:44:40.286777 10292 heartbeater.cc:503] Master 127.23.44.254:45905 was elected leader, sending a full tablet report...
I20260926 20:44:40.281359 10105 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:44:40.289496 10105 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:44:40.329617 10565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.303759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:35849 (local address 127.0.0.1:40766)
0926 20:44:40.316689 (+ 12930us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:40.316771 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:40.317072 (+ 301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:40.321816 (+ 4744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:40.321821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:40.321851 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:40.322337 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:40.322356 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.324077 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.324083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:40.326427 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:40.326441 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.328544 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.328572 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:40.328790 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.328848 (+ 58us) client_negotiation.cc:769] Sending connection context
0926 20:44:40.328999 (+ 151us) client_negotiation.cc:240] Negotiation successful
0926 20:44:40.329202 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12470,"mutex_wait_us":95,"thread_start_us":13578,"threads_started":1}
I20260926 20:44:40.330088 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.304449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39824 (local address 127.23.44.195:41971)
0926 20:44:40.320283 (+ 15834us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:40.320305 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:40.320358 (+ 53us) server_negotiation.cc:407] Connection header received
0926 20:44:40.320512 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:40.320524 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:40.320945 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:40.321185 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:40.322072 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.324385 (+ 2313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.327296 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.328862 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.329052 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.329340 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:40.329483 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 20:44:40.329679 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15604,"thread_start_us":15388,"threads_started":1}
I20260926 20:44:40.330112 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.303777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:41971 (local address 127.0.0.1:39824)
0926 20:44:40.318589 (+ 14812us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:40.318617 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:40.318790 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:40.321197 (+ 2407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:40.321205 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:40.321243 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:40.321789 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:40.321804 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.324548 (+ 2744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.324554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:40.327049 (+ 2495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:40.327064 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.329096 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.329103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:40.329280 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.329329 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 20:44:40.329435 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 20:44:40.329582 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14614,"mutex_wait_us":61,"thread_start_us":14432,"threads_started":1}
I20260926 20:44:40.330742 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.304345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44451 (local address 127.0.0.1:45268)
0926 20:44:40.320626 (+ 16281us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:40.320648 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:40.321519 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:40.322537 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:40.322544 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:40.322572 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:40.322971 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:40.322983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.325016 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.325023 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:40.327387 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:40.327403 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.329684 (+ 2281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.329690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:40.330140 (+ 450us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.330187 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 20:44:40.330273 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 20:44:40.330405 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16117,"mutex_wait_us":62,"thread_start_us":15958,"threads_started":1}
I20260926 20:44:40.330859 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.305012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.23.44.193:44451)
0926 20:44:40.319538 (+ 14526us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:40.319553 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:40.320752 (+ 1199us) server_negotiation.cc:407] Connection header received
0926 20:44:40.321786 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:40.321794 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:40.322035 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:40.322277 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:40.323149 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.324847 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.327628 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.329161 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.329284 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.329550 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:40.330330 (+ 780us) server_negotiation.cc:299] Negotiation successful
0926 20:44:40.330485 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14278,"thread_start_us":14034,"threads_started":1}
I20260926 20:44:40.331066 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.304292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40766 (local address 127.23.44.194:35849)
0926 20:44:40.321199 (+ 16907us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:40.321207 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:40.321233 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:44:40.321326 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:40.321335 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:40.321595 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:40.321807 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:40.322530 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.323926 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.326645 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.328329 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.328488 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.328827 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:40.330512 (+ 1685us) server_negotiation.cc:299] Negotiation successful
0926 20:44:40.330713 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16736,"thread_start_us":16625,"threads_started":1}
I20260926 20:44:40.334597 10227 tablet_service.cc:1467] Processing CreateTablet for tablet d0069a9d40564496a6ffa56a88fc15be (DEFAULT_TABLE table=test-table [id=7ed58788672347ef99a412911093d47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:40.334681 10488 tablet_service.cc:1467] Processing CreateTablet for tablet d0069a9d40564496a6ffa56a88fc15be (DEFAULT_TABLE table=test-table [id=7ed58788672347ef99a412911093d47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:40.334681 10358 tablet_service.cc:1467] Processing CreateTablet for tablet d0069a9d40564496a6ffa56a88fc15be (DEFAULT_TABLE table=test-table [id=7ed58788672347ef99a412911093d47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:40.336073 10227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0069a9d40564496a6ffa56a88fc15be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:40.336124 10358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0069a9d40564496a6ffa56a88fc15be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:40.336212 10488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0069a9d40564496a6ffa56a88fc15be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:40.349357 10571 tablet_bootstrap.cc:492] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776: Bootstrap starting.
I20260926 20:44:40.353355 10573 tablet_bootstrap.cc:492] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9: Bootstrap starting.
I20260926 20:44:40.354161 10571 tablet_bootstrap.cc:654] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:40.354494 10572 tablet_bootstrap.cc:492] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85: Bootstrap starting.
I20260926 20:44:40.355604 10571 log.cc:824] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:40.356915 10573 tablet_bootstrap.cc:654] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:40.357715 10572 tablet_bootstrap.cc:654] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:40.358161 10573 log.cc:824] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:40.359064 10571 tablet_bootstrap.cc:492] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776: No bootstrap required, opened a new log
I20260926 20:44:40.359200 10572 log.cc:824] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:40.359386 10571 ts_tablet_manager.cc:1397] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260926 20:44:40.361820 10573 tablet_bootstrap.cc:492] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9: No bootstrap required, opened a new log
I20260926 20:44:40.362180 10573 ts_tablet_manager.cc:1397] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 20:44:40.366582 10572 tablet_bootstrap.cc:492] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85: No bootstrap required, opened a new log
I20260926 20:44:40.366879 10572 ts_tablet_manager.cc:1397] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260926 20:44:40.367869 10571 raft_consensus.cc:348] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41971 } } peers { permanent_uuid: "d0461bdf55e3470db62ce437a6c55f85" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35849 } } peers { permanent_uuid: "9767880fc61a482e89019fc59e7d4776" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44451 } }
I20260926 20:44:40.368258 10571 raft_consensus.cc:374] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:40.368396 10571 raft_consensus.cc:729] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9767880fc61a482e89019fc59e7d4776, State: Initialized, Role: FOLLOWER
I20260926 20:44:40.369060 10571 consensus_queue.cc:260] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41971 } } peers { permanent_uuid: "d0461bdf55e3470db62ce437a6c55f85" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35849 } } peers { permanent_uuid: "9767880fc61a482e89019fc59e7d4776" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44451 } }
I20260926 20:44:40.371918 10571 ts_tablet_manager.cc:1428] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260926 20:44:40.375424 10573 raft_consensus.cc:348] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41971 } } peers { permanent_uuid: "d0461bdf55e3470db62ce437a6c55f85" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35849 } } peers { permanent_uuid: "9767880fc61a482e89019fc59e7d4776" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44451 } }
I20260926 20:44:40.375957 10573 raft_consensus.cc:374] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:40.376111 10573 raft_consensus.cc:729] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 595fbd8b3ed54c028e2cff30bbc5c4e9, State: Initialized, Role: FOLLOWER
I20260926 20:44:40.377012 10573 consensus_queue.cc:260] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41971 } } peers { permanent_uuid: "d0461bdf55e3470db62ce437a6c55f85" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35849 } } peers { permanent_uuid: "9767880fc61a482e89019fc59e7d4776" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44451 } }
I20260926 20:44:40.377425 10572 raft_consensus.cc:348] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41971 } } peers { permanent_uuid: "d0461bdf55e3470db62ce437a6c55f85" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35849 } } peers { permanent_uuid: "9767880fc61a482e89019fc59e7d4776" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44451 } }
I20260926 20:44:40.378250 10553 heartbeater.cc:503] Master 127.23.44.254:45905 was elected leader, sending a full tablet report...
I20260926 20:44:40.378938 10573 ts_tablet_manager.cc:1428] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9: Time spent starting tablet: real 0.017s user 0.011s sys 0.004s
I20260926 20:44:40.378006 10572 raft_consensus.cc:374] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:40.379213 10572 raft_consensus.cc:729] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0461bdf55e3470db62ce437a6c55f85, State: Initialized, Role: FOLLOWER
I20260926 20:44:40.379961 10572 consensus_queue.cc:260] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41971 } } peers { permanent_uuid: "d0461bdf55e3470db62ce437a6c55f85" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35849 } } peers { permanent_uuid: "9767880fc61a482e89019fc59e7d4776" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44451 } }
I20260926 20:44:40.381855 10423 heartbeater.cc:503] Master 127.23.44.254:45905 was elected leader, sending a full tablet report...
I20260926 20:44:40.382543 10572 ts_tablet_manager.cc:1428] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
W20260926 20:44:40.489593 10554 tablet.cc:2369] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:44:40.498507 10424 tablet.cc:2369] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:44:40.513363 10293 tablet.cc:2369] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:44:40.595086 10578 raft_consensus.cc:482] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:44:40.595476 10578 raft_consensus.cc:504] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41971 } } peers { permanent_uuid: "d0461bdf55e3470db62ce437a6c55f85" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35849 } } peers { permanent_uuid: "9767880fc61a482e89019fc59e7d4776" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44451 } }
I20260926 20:44:40.597395 10578 leader_election.cc:288] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0461bdf55e3470db62ce437a6c55f85 (127.23.44.194:35849), 9767880fc61a482e89019fc59e7d4776 (127.23.44.193:44451)
I20260926 20:44:40.603466 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.597804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:35849 (local address 127.23.44.195:53885)
0926 20:44:40.598139 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:40.598168 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:40.598341 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:40.598788 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:40.598793 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:40.598817 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:40.598987 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:40.598996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.600103 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.600108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:40.601627 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:40.601651 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.602708 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.602713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:40.602890 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.602913 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 20:44:40.603019 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 20:44:40.603163 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260926 20:44:40.603462 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.597731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:53885 (local address 127.23.44.194:35849)
0926 20:44:40.597951 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:40.597959 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:40.598475 (+ 516us) server_negotiation.cc:407] Connection header received
0926 20:44:40.598542 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:40.598546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:40.598615 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:40.598717 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:40.599161 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.599969 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.601806 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.602551 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.602718 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.602792 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:40.603048 (+ 256us) server_negotiation.cc:299] Negotiation successful
0926 20:44:40.603189 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 20:44:40.604023 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.598122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:51739 (local address 127.23.44.193:44451)
0926 20:44:40.598355 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:40.598362 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:40.598661 (+ 299us) server_negotiation.cc:407] Connection header received
0926 20:44:40.598907 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:40.598917 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:40.598987 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:40.599104 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:40.599644 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.600434 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.602293 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.603114 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.603303 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.603370 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:40.603685 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 20:44:40.603809 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260926 20:44:40.604610 10580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.597829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44451 (local address 127.23.44.195:51739)
0926 20:44:40.598536 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:40.598550 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:40.598708 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:40.599121 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:40.599126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:40.599149 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:40.599483 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:40.599492 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.600626 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.600636 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:40.602097 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:40.602117 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.603297 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.603304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:40.603474 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.603496 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 20:44:40.604223 (+ 727us) client_negotiation.cc:240] Negotiation successful
0926 20:44:40.604340 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":357,"threads_started":1}
I20260926 20:44:40.604453 10378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0069a9d40564496a6ffa56a88fc15be" candidate_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0461bdf55e3470db62ce437a6c55f85" is_pre_election: true
I20260926 20:44:40.605077 10378 raft_consensus.cc:2454] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 595fbd8b3ed54c028e2cff30bbc5c4e9 in term 0.
I20260926 20:44:40.605742 10442 leader_election.cc:302] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595fbd8b3ed54c028e2cff30bbc5c4e9, d0461bdf55e3470db62ce437a6c55f85; no voters:
I20260926 20:44:40.605680 10247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0069a9d40564496a6ffa56a88fc15be" candidate_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9767880fc61a482e89019fc59e7d4776" is_pre_election: true
I20260926 20:44:40.606128 10578 raft_consensus.cc:2781] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:44:40.606175 10247 raft_consensus.cc:2454] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 595fbd8b3ed54c028e2cff30bbc5c4e9 in term 0.
I20260926 20:44:40.606298 10578 raft_consensus.cc:482] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:44:40.606395 10578 raft_consensus.cc:3037] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:40.608843 10578 raft_consensus.cc:504] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41971 } } peers { permanent_uuid: "d0461bdf55e3470db62ce437a6c55f85" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35849 } } peers { permanent_uuid: "9767880fc61a482e89019fc59e7d4776" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44451 } }
I20260926 20:44:40.609498 10578 leader_election.cc:288] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [CANDIDATE]: Term 1 election: Requested vote from peers d0461bdf55e3470db62ce437a6c55f85 (127.23.44.194:35849), 9767880fc61a482e89019fc59e7d4776 (127.23.44.193:44451)
I20260926 20:44:40.610031 10378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0069a9d40564496a6ffa56a88fc15be" candidate_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0461bdf55e3470db62ce437a6c55f85"
I20260926 20:44:40.610024 10247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0069a9d40564496a6ffa56a88fc15be" candidate_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9767880fc61a482e89019fc59e7d4776"
I20260926 20:44:40.610248 10378 raft_consensus.cc:3037] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:40.610260 10247 raft_consensus.cc:3037] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:40.612638 10247 raft_consensus.cc:2454] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 595fbd8b3ed54c028e2cff30bbc5c4e9 in term 1.
I20260926 20:44:40.612746 10378 raft_consensus.cc:2454] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 595fbd8b3ed54c028e2cff30bbc5c4e9 in term 1.
I20260926 20:44:40.613125 10443 leader_election.cc:302] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595fbd8b3ed54c028e2cff30bbc5c4e9, 9767880fc61a482e89019fc59e7d4776; no voters:
I20260926 20:44:40.613488 10578 raft_consensus.cc:2781] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:40.613974 10578 raft_consensus.cc:686] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [term 1 LEADER]: Becoming Leader. State: Replica: 595fbd8b3ed54c028e2cff30bbc5c4e9, State: Running, Role: LEADER
I20260926 20:44:40.614444 10578 consensus_queue.cc:237] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41971 } } peers { permanent_uuid: "d0461bdf55e3470db62ce437a6c55f85" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35849 } } peers { permanent_uuid: "9767880fc61a482e89019fc59e7d4776" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44451 } }
I20260926 20:44:40.619329 10106 catalog_manager.cc:5495] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 595fbd8b3ed54c028e2cff30bbc5c4e9 (127.23.44.195). New cstate: current_term: 1 leader_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595fbd8b3ed54c028e2cff30bbc5c4e9" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 41971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0461bdf55e3470db62ce437a6c55f85" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9767880fc61a482e89019fc59e7d4776" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44451 } health_report { overall_health: UNKNOWN } } }
I20260926 20:44:40.704073 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.697760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.23.44.195:41971)
0926 20:44:40.697973 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:40.697980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:40.698002 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:44:40.698200 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:40.698206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:40.698293 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:40.698467 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:44:40.698875 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.699705 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.701592 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.702063 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.702221 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.702874 (+ 653us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:44:40.703409 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:44:40.703618 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:40.703753 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 20:44:40.703810 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 20:44:40.713030 10378 raft_consensus.cc:1261] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85 [term 1 FOLLOWER]: Refusing update from remote peer 595fbd8b3ed54c028e2cff30bbc5c4e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:44:40.713033 10247 raft_consensus.cc:1261] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776 [term 1 FOLLOWER]: Refusing update from remote peer 595fbd8b3ed54c028e2cff30bbc5c4e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:44:40.714267 10581 consensus_queue.cc:1035] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9767880fc61a482e89019fc59e7d4776" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 44451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:44:40.714681 10578 consensus_queue.cc:1035] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0461bdf55e3470db62ce437a6c55f85" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 35849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:44:40.731454 10584 mvcc.cc:205] Tried to move back new op lower bound from 7333705648988782592 to 7333705648600551424. Current Snapshot: MvccSnapshot[applied={T|T < 7333705648988782592 or (T in {7333705648988782592})}]
I20260926 20:44:40.744540 10106 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:44:40.744892 10106 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:44:40.749295 10106 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b37cc7b2ada94c0b8637d9fb08f01a60: Sending DeleteTablet for 3 replicas of tablet d0069a9d40564496a6ffa56a88fc15be
I20260926 20:44:40.750488 10488 tablet_service.cc:1514] Processing DeleteTablet for tablet d0069a9d40564496a6ffa56a88fc15be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:40 UTC) from {username='slave'} at 127.0.0.1:39824
I20260926 20:44:40.751155 10358 tablet_service.cc:1514] Processing DeleteTablet for tablet d0069a9d40564496a6ffa56a88fc15be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:40 UTC) from {username='slave'} at 127.0.0.1:40766
I20260926 20:44:40.755223 10227 tablet_service.cc:1514] Processing DeleteTablet for tablet d0069a9d40564496a6ffa56a88fc15be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:40 UTC) from {username='slave'} at 127.0.0.1:45268
I20260926 20:44:40.757166 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 10165
I20260926 20:44:40.760632 10595 tablet_replica.cc:330] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85: stopping tablet replica
I20260926 20:44:40.761164 10595 raft_consensus.cc:2229] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:44:40.761238 10596 tablet_replica.cc:330] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776: stopping tablet replica
I20260926 20:44:40.761771 10595 raft_consensus.cc:2258] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:40.762148 10596 raft_consensus.cc:2229] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:44:40.762668 10596 raft_consensus.cc:2258] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:40.763715 10594 tablet_replica.cc:330] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9: stopping tablet replica
I20260926 20:44:40.765376 10596 ts_tablet_manager.cc:1905] T d0069a9d40564496a6ffa56a88fc15be P 9767880fc61a482e89019fc59e7d4776: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:40.765373 10595 ts_tablet_manager.cc:1905] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:40.769332 10594 raft_consensus.cc:2229] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:44:40.770236 10594 raft_consensus.cc:2258] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:40.772893 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.758607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.23.44.193:44451)
0926 20:44:40.758809 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:40.758816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:40.761129 (+ 2313us) server_negotiation.cc:407] Connection header received
0926 20:44:40.761229 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:40.761234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:40.761344 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:40.761483 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:40.763721 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.764874 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.766768 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.767566 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.767688 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.770770 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:40.770781 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:40.770799 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:40.771662 (+ 863us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:40.771727 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:40.771754 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:40.771867 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:40.772268 (+ 401us) server_negotiation.cc:299] Negotiation successful
0926 20:44:40.772399 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:44:40.773722 10595 ts_tablet_manager.cc:1918] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:40.774166 10287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:40.776593 10594 ts_tablet_manager.cc:1905] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:40.779942 10595 log.cc:1196] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/d0069a9d40564496a6ffa56a88fc15be
I20260926 20:44:40.780704 10595 ts_tablet_manager.cc:1939] T d0069a9d40564496a6ffa56a88fc15be P d0461bdf55e3470db62ce437a6c55f85: Deleting consensus metadata
I20260926 20:44:40.783002 10091 catalog_manager.cc:4841] TS d0461bdf55e3470db62ce437a6c55f85 (127.23.44.194:35849): tablet d0069a9d40564496a6ffa56a88fc15be (table test-table [id=7ed58788672347ef99a412911093d47a]) successfully deleted
I20260926 20:44:40.788295 10594 ts_tablet_manager.cc:1918] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:40.788504 10594 log.cc:1196] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/d0069a9d40564496a6ffa56a88fc15be
I20260926 20:44:40.789139 10594 ts_tablet_manager.cc:1939] T d0069a9d40564496a6ffa56a88fc15be P 595fbd8b3ed54c028e2cff30bbc5c4e9: Deleting consensus metadata
I20260926 20:44:40.791304 10090 catalog_manager.cc:4841] TS 595fbd8b3ed54c028e2cff30bbc5c4e9 (127.23.44.195:41971): tablet d0069a9d40564496a6ffa56a88fc15be (table test-table [id=7ed58788672347ef99a412911093d47a]) successfully deleted
I20260926 20:44:40.876009 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 10165
I20260926 20:44:40.895706 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 10296
W20260926 20:44:40.895833 10092 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.44.193:44451 (error 108)
I20260926 20:44:40.898123 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.897379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44451 (local address 127.0.0.1:45292)
0926 20:44:40.897840 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:40.897964 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":222,"threads_started":1}
W20260926 20:44:40.898428 10092 catalog_manager.cc:4584] TS 9767880fc61a482e89019fc59e7d4776 (127.23.44.193:44451): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0069a9d40564496a6ffa56a88fc15be: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44451: connect: Connection refused (error 111)
I20260926 20:44:40.904333 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.896594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40770 (local address 127.23.44.194:35849)
0926 20:44:40.896813 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:40.896819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:40.896841 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:44:40.897035 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:40.897041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:40.897154 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:40.897264 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:40.897877 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.899055 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.900543 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:40.901191 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:40.901346 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:40.902595 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:40.902607 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:40.902626 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:40.903443 (+ 817us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:40.903506 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:40.903531 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:40.903655 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:40.903829 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 20:44:40.903946 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:44:40.905122 10418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:40.952297 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.951727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44451 (local address 127.0.0.1:45300)
0926 20:44:40.951998 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:40.952121 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260926 20:44:40.953621 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:40.953171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44451 (local address 127.0.0.1:45310)
0926 20:44:40.953375 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:40.953463 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 20:44:41.004091 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 10296
I20260926 20:44:41.015262 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:41.014463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44451 (local address 127.0.0.1:45314)
0926 20:44:41.014808 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:41.014968 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260926 20:44:41.017179 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:41.016473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44451 (local address 127.0.0.1:45324)
0926 20:44:41.016762 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:41.016924 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260926 20:44:41.027949 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 10426
I20260926 20:44:41.035494 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:41.029157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39844 (local address 127.23.44.195:41971)
0926 20:44:41.029332 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:41.029336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:41.029351 (+ 15us) server_negotiation.cc:407] Connection header received
0926 20:44:41.029405 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:41.029408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:41.029481 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:41.029610 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:41.030022 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:41.030800 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:41.032093 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:41.032537 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:41.032951 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:41.034030 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:41.034038 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:41.034057 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:41.034696 (+ 639us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:41.034748 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:41.034767 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:41.034902 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:41.035048 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 20:44:41.035150 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:44:41.036376 10548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:41.098140 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:41.097551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44451 (local address 127.0.0.1:45334)
0926 20:44:41.097840 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:41.097951 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260926 20:44:41.099300 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:41.098847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44451 (local address 127.0.0.1:45344)
0926 20:44:41.099052 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:41.099135 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260926 20:44:41.164754 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 10426
I20260926 20:44:41.185217 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 10075
I20260926 20:44:41.187081 10136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:41.297771 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:41.297003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:44451 (local address 127.0.0.1:45358)
0926 20:44:41.297364 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:41.297514 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.193:44451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260926 20:44:41.297777 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 10075
2026-09-26T20:44:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3182 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:44:41 dist-test-slave-4d5t krb5kdc[10606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:44:41 dist-test-slave-4d5t krb5kdc[10606](info): set up 2 sockets
Sep 26 20:44:41 dist-test-slave-4d5t krb5kdc[10606](info): commencing operation
krb5kdc: starting...
W20260926 20:44:42.441591 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:44:42 dist-test-slave-4d5t krb5kdc[10606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:44:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:44:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:44:42.736687 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:35879
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:35879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:43.094751 10619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:43.095100 10619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:43.095196 10619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:43.102578 10619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:44:43.102715 10619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:43.102762 10619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:43.102813 10619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:44:43.102859 10619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:44:43.113034 10619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:35879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:35879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:43.114332 10619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:43.115577 10619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:43.128185 10624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:43.128139 10625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:43.128244 10627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:43.130050 10619 server_base.cc:1034] running on GCE node
I20260926 20:44:43.130918 10619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:43.132719 10619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:43.133976 10619 hybrid_clock.cc:648] HybridClock initialized: now 1790455483133875 us; error 82 us; skew 500 ppm
I20260926 20:44:43.134400 10619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:43 dist-test-slave-4d5t krb5kdc[10606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455483, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:43.138406 10619 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:44:43.140210 10619 webserver.cc:460] Webserver started at http://127.23.44.254:40783/ using document root <none> and password file <none>
I20260926 20:44:43.141016 10619 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:43.141124 10619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:43.141386 10619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:43.144480 10619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "3d24a5bd02414b48820f2e546108674a"
format_stamp: "Formatted at 2026-09-26 20:44:43 on dist-test-slave-4d5t"
server_key: "7a2a9af4f3f86415bec6dce3bd6f30e0"
server_key_iv: "59c44375e692d2e5af8a9a1df8be74dc"
server_key_version: "encryptionkey@0"
I20260926 20:44:43.145342 10619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "3d24a5bd02414b48820f2e546108674a"
format_stamp: "Formatted at 2026-09-26 20:44:43 on dist-test-slave-4d5t"
server_key: "7a2a9af4f3f86415bec6dce3bd6f30e0"
server_key_iv: "59c44375e692d2e5af8a9a1df8be74dc"
server_key_version: "encryptionkey@0"
I20260926 20:44:43.151706 10619 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 20:44:43.156239 10634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:43.158353 10619 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260926 20:44:43.158555 10619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "3d24a5bd02414b48820f2e546108674a"
format_stamp: "Formatted at 2026-09-26 20:44:43 on dist-test-slave-4d5t"
server_key: "7a2a9af4f3f86415bec6dce3bd6f30e0"
server_key_iv: "59c44375e692d2e5af8a9a1df8be74dc"
server_key_version: "encryptionkey@0"
I20260926 20:44:43.158789 10619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:43.186859 10619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:43.191753 10619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:43.192025 10619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:43.213299 10619 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:35879
I20260926 20:44:43.213304 10685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:35879 every 8 connection(s)
I20260926 20:44:43.214969 10619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:44:43.216498 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 10619
I20260926 20:44:43.216738 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:44:43.217312 23731 external_mini_cluster.cc:1428] Setting key 5000b0ded9d24e3f94ecf6c997451aca
I20260926 20:44:43.220345 10686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:44:43 dist-test-slave-4d5t krb5kdc[10606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:44:43.233255 10686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a: Bootstrap starting.
I20260926 20:44:43.237967 10686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:43.239351 10686 log.cc:824] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:43.243434 10686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a: No bootstrap required, opened a new log
I20260926 20:44:43.243957 10689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:43.219492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50964 (local address 127.23.44.254:35879)
0926 20:44:43.220357 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:43.220370 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:43.220444 (+ 74us) server_negotiation.cc:407] Connection header received
0926 20:44:43.222377 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:43.222406 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:43.222841 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:43.223308 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:43.225074 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:43.226535 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:43.227987 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:43.228955 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:43.229446 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:43.234686 (+ 5240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:43.234697 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:43.234714 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:43.234771 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:43.238537 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:43.239453 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:43.239461 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:43.239468 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:43.239729 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:43.240287 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:43.240291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:43.240293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:43.241090 (+ 797us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:43.241439 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:43.241956 (+ 517us) server_negotiation.cc:299] Negotiation successful
0926 20:44:43.242482 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":391,"threads_started":1}
I20260926 20:44:43.251915 10686 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d24a5bd02414b48820f2e546108674a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35879 } }
I20260926 20:44:43.252336 10686 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:43.252415 10686 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d24a5bd02414b48820f2e546108674a, State: Initialized, Role: FOLLOWER
I20260926 20:44:43.253141 10686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d24a5bd02414b48820f2e546108674a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35879 } }
I20260926 20:44:43.253417 10686 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:44:43.253572 10686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:44:43.253791 10686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:43.256094 10686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d24a5bd02414b48820f2e546108674a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35879 } }
I20260926 20:44:43.256675 10686 leader_election.cc:302] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d24a5bd02414b48820f2e546108674a; no voters:
I20260926 20:44:43.257386 10686 leader_election.cc:288] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:44:43.257682 10691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:43.258641 10691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [term 1 LEADER]: Becoming Leader. State: Replica: 3d24a5bd02414b48820f2e546108674a, State: Running, Role: LEADER
I20260926 20:44:43.259397 10691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d24a5bd02414b48820f2e546108674a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35879 } }
I20260926 20:44:43.259856 10686 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:44:43.263700 10693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d24a5bd02414b48820f2e546108674a. Latest consensus state: current_term: 1 leader_uuid: "3d24a5bd02414b48820f2e546108674a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d24a5bd02414b48820f2e546108674a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35879 } } }
I20260926 20:44:43.264142 10693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:43.263303 10692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d24a5bd02414b48820f2e546108674a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d24a5bd02414b48820f2e546108674a" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 35879 } } }
I20260926 20:44:43.264622 10692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:43.265130 10700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:44:43.270676 10700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:44:43.281663 10700 catalog_manager.cc:1347] Generated new cluster ID: c073eebf66fc46e9af08c577a0a4bcea
I20260926 20:44:43.281862 10700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:44:43.302816 10700 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:44:43.304174 10700 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:44:43.316727 10700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a: Generated new TSK 0
I20260926 20:44:43.317752 10700 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.44.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
Entry for principal kudu/127.23.44.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab.
I20260926 20:44:43.399922 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:35879
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:43.762161 10712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:43.762495 10712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:43.762602 10712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:43.771080 10712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:43.771217 10712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:43.771342 10712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:44:43.781908 10712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.193.keytab
--principal=kudu/127.23.44.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:35879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:43.783269 10712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:43.784652 10712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:43.798189 10717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:43.798817 10718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:43.799266 10720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:43.800233 10712 server_base.cc:1034] running on GCE node
I20260926 20:44:43.801219 10712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:43.802261 10712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:43.803520 10712 hybrid_clock.cc:648] HybridClock initialized: now 1790455483803454 us; error 43 us; skew 500 ppm
I20260926 20:44:43.803958 10712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:43 dist-test-slave-4d5t krb5kdc[10606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455483, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:43.808120 10712 init.cc:377] Logged in from keytab as kudu/127.23.44.193@KRBTEST.COM (short username kudu)
I20260926 20:44:43.810212 10712 webserver.cc:460] Webserver started at http://127.23.44.193:40217/ using document root <none> and password file <none>
I20260926 20:44:43.811017 10712 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:43.811136 10712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:43.811532 10712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:43.814908 10712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "93146f4977be4785aa9bb9016db4ca61"
format_stamp: "Formatted at 2026-09-26 20:44:43 on dist-test-slave-4d5t"
server_key: "d9660a98658c0209569a3bf4bb2d5cae"
server_key_iv: "74135d42a5540d21160aefc97c1512d5"
server_key_version: "encryptionkey@0"
I20260926 20:44:43.815779 10712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "93146f4977be4785aa9bb9016db4ca61"
format_stamp: "Formatted at 2026-09-26 20:44:43 on dist-test-slave-4d5t"
server_key: "d9660a98658c0209569a3bf4bb2d5cae"
server_key_iv: "74135d42a5540d21160aefc97c1512d5"
server_key_version: "encryptionkey@0"
I20260926 20:44:43.822155 10712 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260926 20:44:43.826354 10727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:43.827988 10712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260926 20:44:43.828299 10712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "93146f4977be4785aa9bb9016db4ca61"
format_stamp: "Formatted at 2026-09-26 20:44:43 on dist-test-slave-4d5t"
server_key: "d9660a98658c0209569a3bf4bb2d5cae"
server_key_iv: "74135d42a5540d21160aefc97c1512d5"
server_key_version: "encryptionkey@0"
I20260926 20:44:43.828557 10712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:43.868762 10712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:43.873711 10712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:43.874056 10712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:43.875147 10712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:43.876726 10712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:43.876835 10712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:43.877013 10712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:43.877100 10712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:43.917029 10712 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:41037
I20260926 20:44:43.917217 10839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:41037 every 8 connection(s)
I20260926 20:44:43.918581 10712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:44:43.928776 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 10712
I20260926 20:44:43.928974 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:44:43.929515 23731 external_mini_cluster.cc:1428] Setting key f34c20b24fa628237cb011de91077684
Sep 26 20:44:43 dist-test-slave-4d5t krb5kdc[10606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455483, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.193@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:44:43.942302 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:43.921554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:38115 (local address 127.23.44.254:35879)
0926 20:44:43.922076 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:43.922083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:43.923028 (+ 945us) server_negotiation.cc:407] Connection header received
0926 20:44:43.924909 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:43.924915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:43.924990 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:43.925113 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:43.926663 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:43.927399 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:43.929439 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:43.929876 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:43.929990 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:43.933954 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:43.933963 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:43.933970 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:43.934037 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:43.937909 (+ 3872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:43.938709 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:43.938717 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:43.938723 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:43.938877 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:43.939287 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:43.939294 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:43.939300 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:43.939687 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:43.939837 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:43.941324 (+ 1487us) server_negotiation.cc:299] Negotiation successful
0926 20:44:43.941745 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":242,"threads_started":1}
I20260926 20:44:43.943739 10843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:43.922158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:35879 (local address 127.23.44.193:38115)
0926 20:44:43.922861 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:43.922905 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:43.924654 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:43.925371 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:43.925388 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:43.925895 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:43.926474 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:43.926491 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:43.927589 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:43.927593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:43.929284 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:43.929299 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:43.930122 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:43.930128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:43.930330 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:43.931009 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:44:43.931044 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:43.933716 (+ 2672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:43.938078 (+ 4362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:43.938085 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:43.938099 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:43.938566 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:43.938982 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:43.938985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:43.938987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:43.939166 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:43.939949 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:43.939965 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:43.940730 (+ 765us) client_negotiation.cc:769] Sending connection context
0926 20:44:43.941142 (+ 412us) client_negotiation.cc:240] Negotiation successful
0926 20:44:43.941900 (+ 758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":288,"threads_started":1}
I20260926 20:44:43.945622 10840 heartbeater.cc:347] Connected to a master server at 127.23.44.254:35879
I20260926 20:44:43.945987 10840 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:43.946920 10840 heartbeater.cc:511] Master 127.23.44.254:35879 requested a full tablet report, sending...
I20260926 20:44:43.949522 10651 ts_manager.cc:194] Registered new tserver with Master: 93146f4977be4785aa9bb9016db4ca61 (127.23.44.193:41037)
I20260926 20:44:43.951197 10651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.193@KRBTEST.COM'} at 127.23.44.193:38115
WARNING: no policy specified for kudu/127.23.44.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
Entry for principal kudu/127.23.44.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab.
I20260926 20:44:43.992486 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:35879
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:44.336764 10846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:44.337132 10846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:44.337306 10846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:44.345280 10846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:44.345453 10846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:44.345633 10846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:44:44.356000 10846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.194.keytab
--principal=kudu/127.23.44.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:35879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:44.357487 10846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:44.358793 10846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:44.372826 10852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:44.372902 10854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:44.372764 10851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:44.373647 10846 server_base.cc:1034] running on GCE node
I20260926 20:44:44.374279 10846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:44.375280 10846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:44.376511 10846 hybrid_clock.cc:648] HybridClock initialized: now 1790455484376469 us; error 44 us; skew 500 ppm
I20260926 20:44:44.376931 10846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:44 dist-test-slave-4d5t krb5kdc[10606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455484, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:44.380430 10846 init.cc:377] Logged in from keytab as kudu/127.23.44.194@KRBTEST.COM (short username kudu)
I20260926 20:44:44.382054 10846 webserver.cc:460] Webserver started at http://127.23.44.194:37411/ using document root <none> and password file <none>
I20260926 20:44:44.382764 10846 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:44.382900 10846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:44.383244 10846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:44.386334 10846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc"
format_stamp: "Formatted at 2026-09-26 20:44:44 on dist-test-slave-4d5t"
server_key: "a053d48c2fef69a156ef72ee83cdd13d"
server_key_iv: "ec13d46203d21cb882f75d8d58794d91"
server_key_version: "encryptionkey@0"
I20260926 20:44:44.387152 10846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc"
format_stamp: "Formatted at 2026-09-26 20:44:44 on dist-test-slave-4d5t"
server_key: "a053d48c2fef69a156ef72ee83cdd13d"
server_key_iv: "ec13d46203d21cb882f75d8d58794d91"
server_key_version: "encryptionkey@0"
I20260926 20:44:44.393357 10846 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260926 20:44:44.397353 10861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:44.399148 10846 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:44:44.399389 10846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc"
format_stamp: "Formatted at 2026-09-26 20:44:44 on dist-test-slave-4d5t"
server_key: "a053d48c2fef69a156ef72ee83cdd13d"
server_key_iv: "ec13d46203d21cb882f75d8d58794d91"
server_key_version: "encryptionkey@0"
I20260926 20:44:44.399637 10846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:44.423606 10846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:44.428290 10846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:44.428681 10846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:44.429706 10846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:44.431097 10846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:44.431212 10846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:44.431360 10846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:44.431432 10846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:44.469446 10846 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:40381
I20260926 20:44:44.469483 10973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:40381 every 8 connection(s)
I20260926 20:44:44.470798 10846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:44:44.481329 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 10846
I20260926 20:44:44.481657 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:44:44.482173 23731 external_mini_cluster.cc:1428] Setting key 8a79fea605c5438b7cc558c4a9e7fb17
Sep 26 20:44:44 dist-test-slave-4d5t krb5kdc[10606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455484, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.194@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:44:44.497377 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:44.473854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:56145 (local address 127.23.44.254:35879)
0926 20:44:44.474522 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:44.474528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:44.475931 (+ 1403us) server_negotiation.cc:407] Connection header received
0926 20:44:44.478607 (+ 2676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:44.478614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:44.478725 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:44.478856 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:44.481336 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:44.482224 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:44.484666 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:44.485334 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:44.485477 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:44.490923 (+ 5446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:44.490930 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:44.490933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:44.490974 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:44.493768 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:44.494640 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:44.494645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:44.494647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:44.494791 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:44.495277 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:44.495287 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:44.495289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:44.495583 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:44.495754 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:44.496511 (+ 757us) server_negotiation.cc:299] Negotiation successful
0926 20:44:44.496801 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":380,"threads_started":1}
I20260926 20:44:44.498490 10977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:44.474520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:35879 (local address 127.23.44.194:56145)
0926 20:44:44.475650 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:44.475753 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:44.478184 (+ 2431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:44.479144 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:44.479153 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:44.479728 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:44.480973 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:44.481057 (+ 84us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:44.482474 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:44.482480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:44.484345 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:44.484369 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:44.485754 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:44.485763 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:44.486125 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:44.486976 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:44:44.487020 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:44.490594 (+ 3574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:44.493968 (+ 3374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:44.493979 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:44.494007 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:44.494489 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:44.494925 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:44.494930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:44.494934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:44.495138 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:44.495725 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:44.495734 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:44.496120 (+ 386us) client_negotiation.cc:769] Sending connection context
0926 20:44:44.496404 (+ 284us) client_negotiation.cc:240] Negotiation successful
0926 20:44:44.497160 (+ 756us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":454,"threads_started":1}
I20260926 20:44:44.500144 10974 heartbeater.cc:347] Connected to a master server at 127.23.44.254:35879
I20260926 20:44:44.500486 10974 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:44.501499 10974 heartbeater.cc:511] Master 127.23.44.254:35879 requested a full tablet report, sending...
I20260926 20:44:44.503873 10651 ts_manager.cc:194] Registered new tserver with Master: d5fc654c5b8e4fc5b5b533d1c40b12cc (127.23.44.194:40381)
I20260926 20:44:44.505407 10651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.194@KRBTEST.COM'} at 127.23.44.194:56145
WARNING: no policy specified for kudu/127.23.44.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
Entry for principal kudu/127.23.44.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab.
I20260926 20:44:44.550498 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:35879
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:44.914390 10980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:44.914703 10980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:44.914855 10980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:44.922909 10980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:44.923030 10980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:44.923229 10980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:44:44.933792 10980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.195.keytab
--principal=kudu/127.23.44.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:35879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:44.935372 10980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:44.936797 10980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:44.951295 10988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:44.951349 10986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:44.951637 10985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:44.952392 10980 server_base.cc:1034] running on GCE node
I20260926 20:44:44.953130 10980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:44.954326 10980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:44.955768 10980 hybrid_clock.cc:648] HybridClock initialized: now 1790455484955679 us; error 73 us; skew 500 ppm
I20260926 20:44:44.956247 10840 heartbeater.cc:503] Master 127.23.44.254:35879 was elected leader, sending a full tablet report...
I20260926 20:44:44.956389 10980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:44 dist-test-slave-4d5t krb5kdc[10606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455484, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:44.961192 10980 init.cc:377] Logged in from keytab as kudu/127.23.44.195@KRBTEST.COM (short username kudu)
I20260926 20:44:44.963191 10980 webserver.cc:460] Webserver started at http://127.23.44.195:35163/ using document root <none> and password file <none>
I20260926 20:44:44.964248 10980 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:44.964414 10980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:44.964906 10980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:44.969149 10980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "85f8c7677f52497fbb090730ab8cca71"
format_stamp: "Formatted at 2026-09-26 20:44:44 on dist-test-slave-4d5t"
server_key: "359a8619439d09eb58f0384656a97d1f"
server_key_iv: "8cf1d1c0c44b1bc8d281d16801525807"
server_key_version: "encryptionkey@0"
I20260926 20:44:44.970079 10980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "85f8c7677f52497fbb090730ab8cca71"
format_stamp: "Formatted at 2026-09-26 20:44:44 on dist-test-slave-4d5t"
server_key: "359a8619439d09eb58f0384656a97d1f"
server_key_iv: "8cf1d1c0c44b1bc8d281d16801525807"
server_key_version: "encryptionkey@0"
I20260926 20:44:44.976807 10980 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 20:44:44.981675 10995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:44.983378 10980 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:44:44.983578 10980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "85f8c7677f52497fbb090730ab8cca71"
format_stamp: "Formatted at 2026-09-26 20:44:44 on dist-test-slave-4d5t"
server_key: "359a8619439d09eb58f0384656a97d1f"
server_key_iv: "8cf1d1c0c44b1bc8d281d16801525807"
server_key_version: "encryptionkey@0"
I20260926 20:44:44.983805 10980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:45.010452 10980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:45.015933 10980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:45.016364 10980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:45.017580 10980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:45.019080 10980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:45.019189 10980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:45.019345 10980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:45.019428 10980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:45.059454 10980 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:35977
I20260926 20:44:45.059518 11107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:35977 every 8 connection(s)
I20260926 20:44:45.061277 10980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:44:45.070541 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 10980
I20260926 20:44:45.070746 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:44:45.071223 23731 external_mini_cluster.cc:1428] Setting key 1fb0ac3369b723c172da126c7c835735
Sep 26 20:44:45 dist-test-slave-4d5t krb5kdc[10606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455484, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.195@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:44:45.084039 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.064232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:41329 (local address 127.23.44.254:35879)
0926 20:44:45.064887 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:45.064895 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:45.065759 (+ 864us) server_negotiation.cc:407] Connection header received
0926 20:44:45.067734 (+ 1975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:45.067739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:45.067827 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:45.067931 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:45.069655 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.070780 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.073057 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.073635 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.073777 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.077840 (+ 4063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:45.077846 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:45.077851 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:45.077904 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:45.080772 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:45.081695 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:45.081699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:45.081701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:45.081829 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:45.082294 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:45.082298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:45.082300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:45.082599 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:45.082732 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:45.083530 (+ 798us) server_negotiation.cc:299] Negotiation successful
0926 20:44:45.083721 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":275,"threads_started":1}
I20260926 20:44:45.085389 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.064772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:35879 (local address 127.23.44.195:41329)
0926 20:44:45.065589 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:45.065633 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:45.067452 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:45.068152 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:45.068161 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:45.068825 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:45.069454 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:45.069472 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.070976 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.070981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:45.072869 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:45.072886 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.073912 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.073918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:45.074234 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.075016 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:44:45.075054 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:45.077574 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:45.080974 (+ 3400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:45.080983 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:45.081000 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:45.081547 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:45.081961 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:44:45.081965 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:44:45.081967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:44:45.082171 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:44:45.082735 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:45.082742 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:45.083112 (+ 370us) client_negotiation.cc:769] Sending connection context
0926 20:44:45.083379 (+ 267us) client_negotiation.cc:240] Negotiation successful
0926 20:44:45.083829 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":315,"threads_started":1}
I20260926 20:44:45.086947 11108 heartbeater.cc:347] Connected to a master server at 127.23.44.254:35879
I20260926 20:44:45.087345 11108 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:45.088222 11108 heartbeater.cc:511] Master 127.23.44.254:35879 requested a full tablet report, sending...
I20260926 20:44:45.089963 10651 ts_manager.cc:194] Registered new tserver with Master: 85f8c7677f52497fbb090730ab8cca71 (127.23.44.195:35977)
I20260926 20:44:45.090898 10651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.44.195@KRBTEST.COM'} at 127.23.44.195:41329
I20260926 20:44:45.100019 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:44:45.118854 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.104978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50974 (local address 127.23.44.254:35879)
0926 20:44:45.105238 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:45.105249 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:45.105606 (+ 357us) server_negotiation.cc:407] Connection header received
0926 20:44:45.105753 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:45.105757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:45.105853 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:45.105944 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:45.107481 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.108297 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.109695 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.110169 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.110279 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.112887 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:45.112893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:45.112897 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:45.112935 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:45.115846 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:45.116679 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:45.116684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:45.116686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:45.116851 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:45.117318 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:45.117322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:45.117324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:45.117672 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:45.117944 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:45.118271 (+ 327us) server_negotiation.cc:299] Negotiation successful
0926 20:44:45.118461 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:44:45.123497 10651 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:44:45.131582 10651 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:44:45.165580 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.146841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37340 (local address 127.23.44.194:40381)
0926 20:44:45.148081 (+ 1240us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:45.148091 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:45.150270 (+ 2179us) server_negotiation.cc:407] Connection header received
0926 20:44:45.150416 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:45.150428 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:45.150796 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:45.151017 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:45.152291 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.156059 (+ 3768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.161068 (+ 5009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.163249 (+ 2181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.163502 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.163786 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:45.164961 (+ 1175us) server_negotiation.cc:299] Negotiation successful
0926 20:44:45.165197 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1070,"thread_start_us":914,"threads_started":1}
I20260926 20:44:45.165944 11123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.146211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:40381 (local address 127.0.0.1:37340)
0926 20:44:45.148531 (+ 2320us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:45.148577 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:45.148704 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:45.151472 (+ 2768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:45.151484 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:45.151529 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:45.152029 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:45.152045 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.156326 (+ 4281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.156336 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:45.160768 (+ 4432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:45.160789 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.163994 (+ 3205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.164002 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:45.164606 (+ 604us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.164686 (+ 80us) client_negotiation.cc:769] Sending connection context
0926 20:44:45.164864 (+ 178us) client_negotiation.cc:240] Negotiation successful
0926 20:44:45.165116 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2176,"mutex_wait_us":103,"thread_start_us":2832,"threads_started":1}
I20260926 20:44:45.169485 11121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.145803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:35977 (local address 127.0.0.1:39702)
0926 20:44:45.147304 (+ 1501us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:45.147347 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:45.147507 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:45.160136 (+ 12629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:45.160148 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:45.160228 (+ 80us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:45.161110 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:45.161126 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.164087 (+ 2961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.164094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:45.166676 (+ 2582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:45.166690 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.168536 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.168541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:45.168796 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.168834 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 20:44:45.168934 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 20:44:45.169089 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1260,"mutex_wait_us":72,"thread_start_us":1116,"threads_started":1}
I20260926 20:44:45.169534 11122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.146114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39702 (local address 127.23.44.195:35977)
0926 20:44:45.159254 (+ 13140us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:45.159264 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:45.159302 (+ 38us) server_negotiation.cc:407] Connection header received
0926 20:44:45.159414 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:45.159423 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:45.159717 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:45.159961 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:45.161374 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.163268 (+ 1894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.166881 (+ 3613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.167966 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.168127 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.168400 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:45.169006 (+ 606us) server_negotiation.cc:299] Negotiation successful
0926 20:44:45.169183 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12940,"thread_start_us":12803,"threads_started":1}
I20260926 20:44:45.170442 11126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.148813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43134 (local address 127.23.44.193:41037)
0926 20:44:45.160937 (+ 12124us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:45.160948 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:45.160980 (+ 32us) server_negotiation.cc:407] Connection header received
0926 20:44:45.161083 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:45.161090 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:45.161360 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:45.161612 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:45.162231 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.163722 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.166800 (+ 3078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.168166 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.168315 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.168591 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:45.169818 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0926 20:44:45.170007 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11880,"thread_start_us":11612,"threads_started":1}
I20260926 20:44:45.170940 11125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.148347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:41037 (local address 127.0.0.1:43134)
0926 20:44:45.158314 (+ 9967us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:45.158338 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:45.158463 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:45.161629 (+ 3166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:45.161634 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:45.161671 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:45.162052 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:45.162065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.163898 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.163907 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:45.166611 (+ 2704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:45.166626 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.168718 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.168725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:45.168888 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.168928 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 20:44:45.169000 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 20:44:45.170690 (+ 1690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9827,"mutex_wait_us":191,"thread_start_us":10199,"threads_started":1}
I20260926 20:44:45.170429 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 0a0b139962b345f9808047d55255a71a (DEFAULT_TABLE table=test-table [id=62870aa2e6b844d994bf006ac0c6d125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:45.172096 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a0b139962b345f9808047d55255a71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:45.175192 10775 tablet_service.cc:1467] Processing CreateTablet for tablet 0a0b139962b345f9808047d55255a71a (DEFAULT_TABLE table=test-table [id=62870aa2e6b844d994bf006ac0c6d125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:45.176513 10775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a0b139962b345f9808047d55255a71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:45.179733 11043 tablet_service.cc:1467] Processing CreateTablet for tablet 0a0b139962b345f9808047d55255a71a (DEFAULT_TABLE table=test-table [id=62870aa2e6b844d994bf006ac0c6d125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:45.181345 11043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a0b139962b345f9808047d55255a71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:45.188839 11127 tablet_bootstrap.cc:492] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc: Bootstrap starting.
I20260926 20:44:45.192781 11128 tablet_bootstrap.cc:492] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61: Bootstrap starting.
I20260926 20:44:45.193182 11127 tablet_bootstrap.cc:654] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:45.194358 11127 log.cc:824] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:45.196339 11130 tablet_bootstrap.cc:492] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71: Bootstrap starting.
I20260926 20:44:45.197607 11127 tablet_bootstrap.cc:492] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc: No bootstrap required, opened a new log
I20260926 20:44:45.197835 11128 tablet_bootstrap.cc:654] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:45.197898 11127 ts_tablet_manager.cc:1397] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260926 20:44:45.199378 11128 log.cc:824] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:45.200374 11130 tablet_bootstrap.cc:654] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:45.201684 11130 log.cc:824] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:45.202543 11128 tablet_bootstrap.cc:492] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61: No bootstrap required, opened a new log
I20260926 20:44:45.202844 11128 ts_tablet_manager.cc:1397] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 20:44:45.205060 11130 tablet_bootstrap.cc:492] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71: No bootstrap required, opened a new log
I20260926 20:44:45.205444 11130 ts_tablet_manager.cc:1397] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260926 20:44:45.209285 11127 raft_consensus.cc:348] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85f8c7677f52497fbb090730ab8cca71" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35977 } } peers { permanent_uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 40381 } } peers { permanent_uuid: "93146f4977be4785aa9bb9016db4ca61" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41037 } }
I20260926 20:44:45.209861 11127 raft_consensus.cc:374] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:45.209985 11127 raft_consensus.cc:729] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fc654c5b8e4fc5b5b533d1c40b12cc, State: Initialized, Role: FOLLOWER
I20260926 20:44:45.210762 11127 consensus_queue.cc:260] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85f8c7677f52497fbb090730ab8cca71" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35977 } } peers { permanent_uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 40381 } } peers { permanent_uuid: "93146f4977be4785aa9bb9016db4ca61" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41037 } }
I20260926 20:44:45.210909 11128 raft_consensus.cc:348] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85f8c7677f52497fbb090730ab8cca71" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35977 } } peers { permanent_uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 40381 } } peers { permanent_uuid: "93146f4977be4785aa9bb9016db4ca61" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41037 } }
I20260926 20:44:45.211380 11128 raft_consensus.cc:374] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:45.211494 11128 raft_consensus.cc:729] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93146f4977be4785aa9bb9016db4ca61, State: Initialized, Role: FOLLOWER
I20260926 20:44:45.212212 11128 consensus_queue.cc:260] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85f8c7677f52497fbb090730ab8cca71" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35977 } } peers { permanent_uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 40381 } } peers { permanent_uuid: "93146f4977be4785aa9bb9016db4ca61" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41037 } }
I20260926 20:44:45.213480 10974 heartbeater.cc:503] Master 127.23.44.254:35879 was elected leader, sending a full tablet report...
I20260926 20:44:45.213619 11127 ts_tablet_manager.cc:1428] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260926 20:44:45.218621 11128 ts_tablet_manager.cc:1428] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260926 20:44:45.222790 11130 raft_consensus.cc:348] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85f8c7677f52497fbb090730ab8cca71" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35977 } } peers { permanent_uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 40381 } } peers { permanent_uuid: "93146f4977be4785aa9bb9016db4ca61" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41037 } }
I20260926 20:44:45.223253 11130 raft_consensus.cc:374] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:45.223388 11130 raft_consensus.cc:729] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85f8c7677f52497fbb090730ab8cca71, State: Initialized, Role: FOLLOWER
W20260926 20:44:45.223670 10975 tablet.cc:2369] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:44:45.224267 11130 consensus_queue.cc:260] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85f8c7677f52497fbb090730ab8cca71" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35977 } } peers { permanent_uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 40381 } } peers { permanent_uuid: "93146f4977be4785aa9bb9016db4ca61" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41037 } }
I20260926 20:44:45.225525 11108 heartbeater.cc:503] Master 127.23.44.254:35879 was elected leader, sending a full tablet report...
I20260926 20:44:45.226184 11130 ts_tablet_manager.cc:1428] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71: Time spent starting tablet: real 0.021s user 0.016s sys 0.000s
W20260926 20:44:45.313330 11109 tablet.cc:2369] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:44:45.350785 11135 raft_consensus.cc:482] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:44:45.351089 11135 raft_consensus.cc:504] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85f8c7677f52497fbb090730ab8cca71" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35977 } } peers { permanent_uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 40381 } } peers { permanent_uuid: "93146f4977be4785aa9bb9016db4ca61" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41037 } }
I20260926 20:44:45.352789 11135 leader_election.cc:288] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5fc654c5b8e4fc5b5b533d1c40b12cc (127.23.44.194:40381), 93146f4977be4785aa9bb9016db4ca61 (127.23.44.193:41037)
I20260926 20:44:45.359727 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.353234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:40381 (local address 127.23.44.195:45783)
0926 20:44:45.353573 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:45.353595 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:45.353764 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:45.354155 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:45.354161 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:45.354198 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:45.354429 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:45.354438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.355636 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.355640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:45.357719 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:45.357742 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.359130 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.359134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:45.359286 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.359315 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 20:44:45.359385 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 20:44:45.359506 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260926 20:44:45.359889 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.353328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:45783 (local address 127.23.44.194:40381)
0926 20:44:45.353563 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:45.353569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:45.353826 (+ 257us) server_negotiation.cc:407] Connection header received
0926 20:44:45.353925 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:45.353933 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:45.354017 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:45.354105 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:45.354564 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.355474 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.357940 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.358926 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.359287 (+ 361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.359397 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:45.359474 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 20:44:45.359610 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 20:44:45.361123 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a0b139962b345f9808047d55255a71a" candidate_uuid: "85f8c7677f52497fbb090730ab8cca71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc" is_pre_election: true
I20260926 20:44:45.361445 11126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.353341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:33371 (local address 127.23.44.193:41037)
0926 20:44:45.353755 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:45.353762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:45.354199 (+ 437us) server_negotiation.cc:407] Connection header received
0926 20:44:45.354419 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:45.354423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:45.354515 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:45.354625 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:45.355524 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.356364 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.359031 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.360425 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.360683 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.360760 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:45.361048 (+ 288us) server_negotiation.cc:299] Negotiation successful
0926 20:44:45.361186 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260926 20:44:45.361578 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.353295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:41037 (local address 127.23.44.195:33371)
0926 20:44:45.354052 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:45.354065 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:45.354274 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:45.354770 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:45.354775 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:45.354820 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:45.355277 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:45.355288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.356583 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.356589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:45.358765 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:45.358777 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.360685 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.360691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:45.360870 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.360905 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 20:44:45.361010 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 20:44:45.361160 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"spinlock_wait_cycles":39808,"thread_start_us":444,"threads_started":1}
I20260926 20:44:45.361754 10929 raft_consensus.cc:2454] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85f8c7677f52497fbb090730ab8cca71 in term 0.
I20260926 20:44:45.362581 10996 leader_election.cc:302] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85f8c7677f52497fbb090730ab8cca71, d5fc654c5b8e4fc5b5b533d1c40b12cc; no voters:
I20260926 20:44:45.362699 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a0b139962b345f9808047d55255a71a" candidate_uuid: "85f8c7677f52497fbb090730ab8cca71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93146f4977be4785aa9bb9016db4ca61" is_pre_election: true
I20260926 20:44:45.363080 11135 raft_consensus.cc:2781] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:44:45.363267 11135 raft_consensus.cc:482] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:44:45.363276 10795 raft_consensus.cc:2454] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85f8c7677f52497fbb090730ab8cca71 in term 0.
I20260926 20:44:45.363438 11135 raft_consensus.cc:3037] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:45.366245 11135 raft_consensus.cc:504] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85f8c7677f52497fbb090730ab8cca71" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35977 } } peers { permanent_uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 40381 } } peers { permanent_uuid: "93146f4977be4785aa9bb9016db4ca61" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41037 } }
I20260926 20:44:45.366962 11135 leader_election.cc:288] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [CANDIDATE]: Term 1 election: Requested vote from peers d5fc654c5b8e4fc5b5b533d1c40b12cc (127.23.44.194:40381), 93146f4977be4785aa9bb9016db4ca61 (127.23.44.193:41037)
I20260926 20:44:45.367509 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a0b139962b345f9808047d55255a71a" candidate_uuid: "85f8c7677f52497fbb090730ab8cca71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc"
I20260926 20:44:45.367534 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a0b139962b345f9808047d55255a71a" candidate_uuid: "85f8c7677f52497fbb090730ab8cca71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93146f4977be4785aa9bb9016db4ca61"
I20260926 20:44:45.367746 10795 raft_consensus.cc:3037] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:45.367733 10929 raft_consensus.cc:3037] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:45.370275 10795 raft_consensus.cc:2454] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85f8c7677f52497fbb090730ab8cca71 in term 1.
I20260926 20:44:45.370283 10929 raft_consensus.cc:2454] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85f8c7677f52497fbb090730ab8cca71 in term 1.
I20260926 20:44:45.370841 10996 leader_election.cc:302] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 85f8c7677f52497fbb090730ab8cca71, 93146f4977be4785aa9bb9016db4ca61, d5fc654c5b8e4fc5b5b533d1c40b12cc; no voters:
I20260926 20:44:45.371208 11135 raft_consensus.cc:2781] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:45.371822 11135 raft_consensus.cc:686] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [term 1 LEADER]: Becoming Leader. State: Replica: 85f8c7677f52497fbb090730ab8cca71, State: Running, Role: LEADER
I20260926 20:44:45.372417 11135 consensus_queue.cc:237] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85f8c7677f52497fbb090730ab8cca71" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35977 } } peers { permanent_uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 40381 } } peers { permanent_uuid: "93146f4977be4785aa9bb9016db4ca61" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41037 } }
I20260926 20:44:45.377466 10651 catalog_manager.cc:5495] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85f8c7677f52497fbb090730ab8cca71 (127.23.44.195). New cstate: current_term: 1 leader_uuid: "85f8c7677f52497fbb090730ab8cca71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85f8c7677f52497fbb090730ab8cca71" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 35977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 40381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93146f4977be4785aa9bb9016db4ca61" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41037 } health_report { overall_health: UNKNOWN } } }
I20260926 20:44:45.406713 11122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.400028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36544 (local address 127.23.44.195:35977)
0926 20:44:45.400299 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:45.400305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:45.400329 (+ 24us) server_negotiation.cc:407] Connection header received
0926 20:44:45.400599 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:45.400607 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:45.400727 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:45.400958 (+ 231us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:44:45.401445 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.402543 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.404281 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.404886 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.404994 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.405548 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:44:45.406066 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:44:45.406176 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:45.406335 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 20:44:45.406409 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:44:45.415906 10929 raft_consensus.cc:1261] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc [term 1 FOLLOWER]: Refusing update from remote peer 85f8c7677f52497fbb090730ab8cca71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:44:45.415902 10795 raft_consensus.cc:1261] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61 [term 1 FOLLOWER]: Refusing update from remote peer 85f8c7677f52497fbb090730ab8cca71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:44:45.417009 11135 consensus_queue.cc:1035] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93146f4977be4785aa9bb9016db4ca61" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 41037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:44:45.417402 11137 consensus_queue.cc:1035] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc654c5b8e4fc5b5b533d1c40b12cc" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 40381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 20:44:45.421501 10841 tablet.cc:2369] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:44:45.430437 11140 mvcc.cc:205] Tried to move back new op lower bound from 7333705668250578944 to 7333705668089827328. Current Snapshot: MvccSnapshot[applied={T|T < 7333705668250578944}]
I20260926 20:44:45.449337 10649 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:44:45.449640 10649 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:44:45.453716 10649 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d24a5bd02414b48820f2e546108674a: Sending DeleteTablet for 3 replicas of tablet 0a0b139962b345f9808047d55255a71a
I20260926 20:44:45.455057 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a0b139962b345f9808047d55255a71a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:45 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:37340
I20260926 20:44:45.455165 10775 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a0b139962b345f9808047d55255a71a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:45 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:43134
I20260926 20:44:45.455369 11043 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a0b139962b345f9808047d55255a71a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:45 UTC) from {username='kudu', principal='kudu/127.23.44.254@KRBTEST.COM'} at 127.0.0.1:39702
I20260926 20:44:45.457355 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 10712
I20260926 20:44:45.467715 11151 tablet_replica.cc:330] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61: stopping tablet replica
I20260926 20:44:45.468375 11151 raft_consensus.cc:2229] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:44:45.469118 11151 raft_consensus.cc:2258] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:45.471650 11151 ts_tablet_manager.cc:1905] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:45.480306 11151 ts_tablet_manager.cc:1918] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:45.480542 11151 log.cc:1196] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/0a0b139962b345f9808047d55255a71a
I20260926 20:44:45.481213 11151 ts_tablet_manager.cc:1939] T 0a0b139962b345f9808047d55255a71a P 93146f4977be4785aa9bb9016db4ca61: Deleting consensus metadata
I20260926 20:44:45.483635 10638 catalog_manager.cc:4841] TS 93146f4977be4785aa9bb9016db4ca61 (127.23.44.193:41037): tablet 0a0b139962b345f9808047d55255a71a (table test-table [id=62870aa2e6b844d994bf006ac0c6d125]) successfully deleted
I20260926 20:44:45.485886 11152 tablet_replica.cc:330] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71: stopping tablet replica
I20260926 20:44:45.486179 11150 tablet_replica.cc:330] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc: stopping tablet replica
I20260926 20:44:45.486429 11152 raft_consensus.cc:2229] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:44:45.486702 11150 raft_consensus.cc:2229] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:44:45.487373 11150 raft_consensus.cc:2258] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:45.487447 11152 raft_consensus.cc:2258] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:45.489892 11150 ts_tablet_manager.cc:1905] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:45.490036 11152 ts_tablet_manager.cc:1905] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:45.497989 11150 ts_tablet_manager.cc:1918] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:45.498210 11152 ts_tablet_manager.cc:1918] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:45.498267 11150 log.cc:1196] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/0a0b139962b345f9808047d55255a71a
I20260926 20:44:45.498394 11152 log.cc:1196] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/0a0b139962b345f9808047d55255a71a
I20260926 20:44:45.498891 11152 ts_tablet_manager.cc:1939] T 0a0b139962b345f9808047d55255a71a P 85f8c7677f52497fbb090730ab8cca71: Deleting consensus metadata
I20260926 20:44:45.499046 11150 ts_tablet_manager.cc:1939] T 0a0b139962b345f9808047d55255a71a P d5fc654c5b8e4fc5b5b533d1c40b12cc: Deleting consensus metadata
I20260926 20:44:45.501034 10635 catalog_manager.cc:4841] TS d5fc654c5b8e4fc5b5b533d1c40b12cc (127.23.44.194:40381): tablet 0a0b139962b345f9808047d55255a71a (table test-table [id=62870aa2e6b844d994bf006ac0c6d125]) successfully deleted
I20260926 20:44:45.501248 10638 catalog_manager.cc:4841] TS 85f8c7677f52497fbb090730ab8cca71 (127.23.44.195:35977): tablet 0a0b139962b345f9808047d55255a71a (table test-table [id=62870aa2e6b844d994bf006ac0c6d125]) successfully deleted
Sep 26 20:44:45 dist-test-slave-4d5t krb5kdc[10606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.193@KRBTEST.COM
I20260926 20:44:45.518561 11126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.459045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.23.44.193:41037)
0926 20:44:45.459286 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:45.459296 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:45.485144 (+ 25848us) server_negotiation.cc:407] Connection header received
0926 20:44:45.485372 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:45.485380 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:45.485495 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:45.485599 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:45.488010 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.489345 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.500754 (+ 11409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.501489 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.501623 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.511132 (+ 9509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:45.511149 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:45.511163 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:45.511208 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:45.515366 (+ 4158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:45.516239 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:45.516246 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:45.516249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:45.516416 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:45.516863 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:45.516878 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:45.516881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:45.517281 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:45.517605 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:45.517942 (+ 337us) server_negotiation.cc:299] Negotiation successful
0926 20:44:45.518190 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:44:45.519224 10835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:45.614619 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 10712
I20260926 20:44:45.632787 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 10846
Sep 26 20:44:45 dist-test-slave-4d5t krb5kdc[10606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.194@KRBTEST.COM
I20260926 20:44:45.652166 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.633711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34190 (local address 127.23.44.194:40381)
0926 20:44:45.633923 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:45.633930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:45.633954 (+ 24us) server_negotiation.cc:407] Connection header received
0926 20:44:45.634020 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:45.634026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:45.634108 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:45.634226 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:45.635508 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.636794 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.638152 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.638779 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.638916 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.644476 (+ 5560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:45.644498 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:45.644520 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:45.644603 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:45.648655 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:45.649864 (+ 1209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:45.649874 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:45.649896 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:45.650063 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:45.650505 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:45.650509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:45.650513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:45.651019 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:45.651344 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:45.651538 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 20:44:45.651772 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:44:45.652969 10969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:45.745743 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 10846
I20260926 20:44:45.766155 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 10980
Sep 26 20:44:45 dist-test-slave-4d5t krb5kdc[10606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.195@KRBTEST.COM
I20260926 20:44:45.786096 11122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:45.767068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36548 (local address 127.23.44.195:35977)
0926 20:44:45.767241 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:45.767246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:45.767265 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:44:45.767405 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:45.767409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:45.767470 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:45.767554 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:45.769434 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.770261 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.772237 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:45.772890 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:45.773028 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:45.779467 (+ 6439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:45.779484 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:45.779501 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:45.779553 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:45.782997 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:45.783826 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:45.783834 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:45.783836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:45.783992 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:45.784422 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:45.784428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:45.784431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:45.784885 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:45.785252 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:45.785473 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 20:44:45.785712 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 20:44:45.786820 11103 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:45.918015 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 10980
I20260926 20:44:45.941208 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 10619
I20260926 20:44:45.942420 10681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:46.055891 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 10619
2026-09-26T20:44:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4760 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T20:44:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:44:46Z Disabled control of system clock
I20260926 20:44:46.141120 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:43197
--webserver_interface=127.23.44.254
--webserver_port=0
--builtin_ntp_servers=127.23.44.212:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:43197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 20:44:46.491990 11161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:46.492328 11161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:46.492470 11161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:46.500173 11161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:44:46.500303 11161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:46.500376 11161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:46.500475 11161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:44:46.500584 11161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:44:46.510403 11161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:43197
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:43197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:46.512235 11161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:46.514513 11161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:46.528818 11166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:46.528841 11167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:46.529589 11169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:46.530086 11161 server_base.cc:1034] running on GCE node
I20260926 20:44:46.531224 11161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:46.532917 11161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:46.534360 11161 hybrid_clock.cc:648] HybridClock initialized: now 1790455486534251 us; error 67 us; skew 500 ppm
I20260926 20:44:46.535110 11161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:44:46.537964 11161 webserver.cc:460] Webserver started at http://127.23.44.254:33851/ using document root <none> and password file <none>
I20260926 20:44:46.538760 11161 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:46.538923 11161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:46.539275 11161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:46.542631 11161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "4b27646ca790411bbbe5ea6b0c962e6e"
format_stamp: "Formatted at 2026-09-26 20:44:46 on dist-test-slave-4d5t"
server_key: "84cc3aae13aef818bb4b0bcbc71d78ae"
server_key_iv: "d9450245b0d2ba9b41493fcc11403ba3"
server_key_version: "encryptionkey@0"
I20260926 20:44:46.543571 11161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "4b27646ca790411bbbe5ea6b0c962e6e"
format_stamp: "Formatted at 2026-09-26 20:44:46 on dist-test-slave-4d5t"
server_key: "84cc3aae13aef818bb4b0bcbc71d78ae"
server_key_iv: "d9450245b0d2ba9b41493fcc11403ba3"
server_key_version: "encryptionkey@0"
I20260926 20:44:46.550058 11161 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 20:44:46.553864 11175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:46.555482 11161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:44:46.555732 11161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "4b27646ca790411bbbe5ea6b0c962e6e"
format_stamp: "Formatted at 2026-09-26 20:44:46 on dist-test-slave-4d5t"
server_key: "84cc3aae13aef818bb4b0bcbc71d78ae"
server_key_iv: "d9450245b0d2ba9b41493fcc11403ba3"
server_key_version: "encryptionkey@0"
I20260926 20:44:46.556221 11161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:46.575136 11161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:46.576097 11161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:46.576408 11161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:46.598888 11161 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:43197
I20260926 20:44:46.598941 11226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:43197 every 8 connection(s)
I20260926 20:44:46.600353 11161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:44:46.604651 11227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:46.608474 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 11161
I20260926 20:44:46.608704 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:44:46.609246 23731 external_mini_cluster.cc:1428] Setting key aee610843984d232916121e1ed375284
I20260926 20:44:46.619235 11227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e: Bootstrap starting.
I20260926 20:44:46.622731 11227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:46.623927 11227 log.cc:824] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:46.624240 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:46.611971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.23.44.254:43197)
0926 20:44:46.613423 (+ 1452us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:46.613438 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:46.613489 (+ 51us) server_negotiation.cc:407] Connection header received
0926 20:44:46.615270 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:46.615295 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:46.615860 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:46.616299 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:46.616948 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:46.618243 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:46.619731 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:46.620409 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:46.620646 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:46.621884 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:46.621892 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:46.621905 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:46.622455 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:46.622546 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:46.622567 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:46.622726 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:46.622931 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 20:44:46.623262 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":553,"threads_started":1}
I20260926 20:44:46.627624 11227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e: No bootstrap required, opened a new log
I20260926 20:44:46.635238 11227 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b27646ca790411bbbe5ea6b0c962e6e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 43197 } }
I20260926 20:44:46.635677 11227 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:46.635790 11227 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b27646ca790411bbbe5ea6b0c962e6e, State: Initialized, Role: FOLLOWER
I20260926 20:44:46.636418 11227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b27646ca790411bbbe5ea6b0c962e6e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 43197 } }
I20260926 20:44:46.636741 11227 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:44:46.636895 11227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:44:46.637090 11227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:46.639304 11227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b27646ca790411bbbe5ea6b0c962e6e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 43197 } }
I20260926 20:44:46.639935 11227 leader_election.cc:302] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b27646ca790411bbbe5ea6b0c962e6e; no voters:
I20260926 20:44:46.640707 11227 leader_election.cc:288] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:44:46.640852 11232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:46.641618 11232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [term 1 LEADER]: Becoming Leader. State: Replica: 4b27646ca790411bbbe5ea6b0c962e6e, State: Running, Role: LEADER
I20260926 20:44:46.642293 11232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b27646ca790411bbbe5ea6b0c962e6e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 43197 } }
I20260926 20:44:46.642962 11227 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:44:46.645635 11233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b27646ca790411bbbe5ea6b0c962e6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b27646ca790411bbbe5ea6b0c962e6e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 43197 } } }
I20260926 20:44:46.645881 11234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b27646ca790411bbbe5ea6b0c962e6e. Latest consensus state: current_term: 1 leader_uuid: "4b27646ca790411bbbe5ea6b0c962e6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b27646ca790411bbbe5ea6b0c962e6e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 43197 } } }
I20260926 20:44:46.646085 11233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:46.646158 11234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:46.648895 11237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:44:46.655282 11237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:44:46.665630 11237 catalog_manager.cc:1347] Generated new cluster ID: fb7d492805154f82992a6257e985022e
I20260926 20:44:46.665768 11237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:44:46.679260 11237 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:44:46.680801 11237 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:44:46.686975 11237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e: Generated new TSK 0
I20260926 20:44:46.687891 11237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:44:46.698625 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.193:0
--local_ip_for_outbound_sockets=127.23.44.193
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:43197
--builtin_ntp_servers=127.23.44.212:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 20:44:47.100965 11251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:47.101275 11251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:47.101454 11251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:47.109894 11251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:47.110024 11251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:47.110211 11251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.193
I20260926 20:44:47.120540 11251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.44.193
--webserver_port=0
--tserver_master_addrs=127.23.44.254:43197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.193
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:47.121999 11251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:47.124330 11251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:47.139761 11257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:47.139842 11259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:47.139797 11256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:47.141376 11251 server_base.cc:1034] running on GCE node
I20260926 20:44:47.142221 11251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:47.143505 11251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:47.144747 11251 hybrid_clock.cc:648] HybridClock initialized: now 1790455487144674 us; error 100 us; skew 500 ppm
I20260926 20:44:47.145170 11251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:44:47.148020 11251 webserver.cc:460] Webserver started at http://127.23.44.193:40635/ using document root <none> and password file <none>
I20260926 20:44:47.148999 11251 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:47.149171 11251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:47.149539 11251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:47.153724 11251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/data/instance:
uuid: "9f630cec46f6490c983e1771976c0c50"
format_stamp: "Formatted at 2026-09-26 20:44:47 on dist-test-slave-4d5t"
server_key: "a6b83f2ed2fbd16c5a65529cfdd93383"
server_key_iv: "c62caea9c939da1c560b038a041639ac"
server_key_version: "encryptionkey@0"
I20260926 20:44:47.154829 11251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance:
uuid: "9f630cec46f6490c983e1771976c0c50"
format_stamp: "Formatted at 2026-09-26 20:44:47 on dist-test-slave-4d5t"
server_key: "a6b83f2ed2fbd16c5a65529cfdd93383"
server_key_iv: "c62caea9c939da1c560b038a041639ac"
server_key_version: "encryptionkey@0"
I20260926 20:44:47.161813 11251 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260926 20:44:47.166558 11265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:47.168232 11251 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260926 20:44:47.168480 11251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/wal
uuid: "9f630cec46f6490c983e1771976c0c50"
format_stamp: "Formatted at 2026-09-26 20:44:47 on dist-test-slave-4d5t"
server_key: "a6b83f2ed2fbd16c5a65529cfdd93383"
server_key_iv: "c62caea9c939da1c560b038a041639ac"
server_key_version: "encryptionkey@0"
I20260926 20:44:47.168816 11251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:47.197319 11251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:47.198405 11251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:47.198824 11251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:47.200163 11251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:47.202225 11251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:47.202394 11251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:47.202579 11251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:47.202696 11251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:47.248385 11251 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.193:40013
I20260926 20:44:47.248538 11377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.193:40013 every 8 connection(s)
I20260926 20:44:47.250248 11251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/data/info.pb
I20260926 20:44:47.257001 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 11251
I20260926 20:44:47.257220 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/wal/instance
I20260926 20:44:47.257640 23731 external_mini_cluster.cc:1428] Setting key 8c921504f8d1fb46704f78b6d7f319a9
I20260926 20:44:47.260519 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.194:0
--local_ip_for_outbound_sockets=127.23.44.194
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:43197
--builtin_ntp_servers=127.23.44.212:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 20:44:47.267572 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:47.253605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.193:45757 (local address 127.23.44.254:43197)
0926 20:44:47.254126 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:47.254131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:47.255530 (+ 1399us) server_negotiation.cc:407] Connection header received
0926 20:44:47.258259 (+ 2729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:47.258266 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:47.258390 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:47.258544 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:47.259980 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:47.261020 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:47.263905 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:47.264411 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:47.264539 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:47.266488 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:47.266495 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:47.266499 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:47.266527 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:47.266592 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:47.266608 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:47.266755 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:47.267146 (+ 391us) server_negotiation.cc:299] Negotiation successful
0926 20:44:47.267256 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":278,"threads_started":1}
I20260926 20:44:47.269012 11381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:47.254318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:43197 (local address 127.23.44.193:45757)
0926 20:44:47.255346 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:47.255385 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:47.257783 (+ 2398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:47.258987 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:47.258998 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:47.259111 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:47.259770 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:47.259787 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:47.261313 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:47.261322 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:47.263687 (+ 2365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:47.263708 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:47.264708 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:47.264717 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:47.265079 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:47.265952 (+ 873us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:44:47.265958 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:47.266018 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:44:47.266144 (+ 126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:47.266759 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:47.266775 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:47.266787 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 20:44:47.267016 (+ 229us) client_negotiation.cc:240] Negotiation successful
0926 20:44:47.267509 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":352,"threads_started":1}
I20260926 20:44:47.270793 11378 heartbeater.cc:347] Connected to a master server at 127.23.44.254:43197
I20260926 20:44:47.271210 11378 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:47.272287 11378 heartbeater.cc:511] Master 127.23.44.254:43197 requested a full tablet report, sending...
I20260926 20:44:47.275158 11192 ts_manager.cc:194] Registered new tserver with Master: 9f630cec46f6490c983e1771976c0c50 (127.23.44.193:40013)
I20260926 20:44:47.276816 11192 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.44.193:45757
W20260926 20:44:47.648165 11382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:47.648651 11382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:47.648823 11382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:47.657416 11382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:47.657552 11382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:47.657770 11382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.194
I20260926 20:44:47.669194 11382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.44.194
--webserver_port=0
--tserver_master_addrs=127.23.44.254:43197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.194
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:47.670646 11382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:47.672703 11382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:47.688627 11387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:47.688486 11388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:47.688727 11390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:47.690110 11382 server_base.cc:1034] running on GCE node
I20260926 20:44:47.690907 11382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:47.691960 11382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:47.693194 11382 hybrid_clock.cc:648] HybridClock initialized: now 1790455487693147 us; error 39 us; skew 500 ppm
I20260926 20:44:47.693602 11382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:44:47.696347 11382 webserver.cc:460] Webserver started at http://127.23.44.194:34433/ using document root <none> and password file <none>
I20260926 20:44:47.697185 11382 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:47.697321 11382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:47.697679 11382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:47.701004 11382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/data/instance:
uuid: "59e6550dcaf54b41a6ba97c140c03f8e"
format_stamp: "Formatted at 2026-09-26 20:44:47 on dist-test-slave-4d5t"
server_key: "7e5efb5e0123a8c6de9a08cc4d0f3a0f"
server_key_iv: "b8decf764d5b457aae49f5713872df87"
server_key_version: "encryptionkey@0"
I20260926 20:44:47.701963 11382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance:
uuid: "59e6550dcaf54b41a6ba97c140c03f8e"
format_stamp: "Formatted at 2026-09-26 20:44:47 on dist-test-slave-4d5t"
server_key: "7e5efb5e0123a8c6de9a08cc4d0f3a0f"
server_key_iv: "b8decf764d5b457aae49f5713872df87"
server_key_version: "encryptionkey@0"
I20260926 20:44:47.707777 11382 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:44:47.712322 11396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:47.714545 11382 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260926 20:44:47.714895 11382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/wal
uuid: "59e6550dcaf54b41a6ba97c140c03f8e"
format_stamp: "Formatted at 2026-09-26 20:44:47 on dist-test-slave-4d5t"
server_key: "7e5efb5e0123a8c6de9a08cc4d0f3a0f"
server_key_iv: "b8decf764d5b457aae49f5713872df87"
server_key_version: "encryptionkey@0"
I20260926 20:44:47.715219 11382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:47.731788 11382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:47.732726 11382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:47.733067 11382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:47.734177 11382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:47.735771 11382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:47.735862 11382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:47.735939 11382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:47.736047 11382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:47.775612 11382 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.194:33295
I20260926 20:44:47.775660 11508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.194:33295 every 8 connection(s)
I20260926 20:44:47.777455 11382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/data/info.pb
I20260926 20:44:47.787339 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 11382
I20260926 20:44:47.787670 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/wal/instance
I20260926 20:44:47.788149 23731 external_mini_cluster.cc:1428] Setting key 5474d1742b0982ecf4b022e667251025
I20260926 20:44:47.792524 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:47.780646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.194:56945 (local address 127.23.44.254:43197)
0926 20:44:47.781211 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:47.781216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:47.782237 (+ 1021us) server_negotiation.cc:407] Connection header received
0926 20:44:47.784274 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:47.784281 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:47.784398 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:47.784537 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:47.785821 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:47.786651 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:47.788686 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:47.789409 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:47.789569 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:47.791307 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:47.791316 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:47.791321 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:47.791348 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:47.791419 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:47.791439 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:47.791719 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:47.791982 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 20:44:47.792101 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":305,"threads_started":1}
I20260926 20:44:47.792624 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.44.195:0
--local_ip_for_outbound_sockets=127.23.44.195
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:43197
--builtin_ntp_servers=127.23.44.212:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 20:44:47.793651 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:47.781079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:43197 (local address 127.23.44.194:56945)
0926 20:44:47.782065 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:47.782104 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:47.783958 (+ 1854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:47.784845 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:47.784854 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:47.784963 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:47.785628 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:47.785647 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:47.786832 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:47.786839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:47.788457 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:47.788472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:47.789678 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:47.789687 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:47.789960 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:47.790802 (+ 842us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:44:47.790806 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:47.790880 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:44:47.791002 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:47.791710 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:47.791719 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:47.791726 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 20:44:47.791913 (+ 187us) client_negotiation.cc:240] Negotiation successful
0926 20:44:47.792258 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":314,"threads_started":1}
I20260926 20:44:47.795517 11509 heartbeater.cc:347] Connected to a master server at 127.23.44.254:43197
I20260926 20:44:47.795849 11509 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:47.796746 11509 heartbeater.cc:511] Master 127.23.44.254:43197 requested a full tablet report, sending...
I20260926 20:44:47.798444 11192 ts_manager.cc:194] Registered new tserver with Master: 59e6550dcaf54b41a6ba97c140c03f8e (127.23.44.194:33295)
I20260926 20:44:47.799718 11192 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.44.194:56945
W20260926 20:44:48.156487 11513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:48.156857 11513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:48.156956 11513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:48.164469 11513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:48.164636 11513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:48.164769 11513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.44.195
I20260926 20:44:48.174644 11513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.44.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.44.195
--webserver_port=0
--tserver_master_addrs=127.23.44.254:43197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.44.195
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:48.175946 11513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:48.177716 11513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:48.191951 11518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:48.192159 11519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:48.191956 11521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:48.197953 11513 server_base.cc:1034] running on GCE node
I20260926 20:44:48.198743 11513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:48.199895 11513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:48.201260 11513 hybrid_clock.cc:648] HybridClock initialized: now 1790455488201175 us; error 108 us; skew 500 ppm
I20260926 20:44:48.201717 11513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 20:44:48.204227 11513 webserver.cc:460] Webserver started at http://127.23.44.195:44361/ using document root <none> and password file <none>
I20260926 20:44:48.205049 11513 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:48.205194 11513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:48.205523 11513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:48.208503 11513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/data/instance:
uuid: "db58290d858d4788ba38b140180080ff"
format_stamp: "Formatted at 2026-09-26 20:44:48 on dist-test-slave-4d5t"
server_key: "c81704b1b2c8d406c5e4699bc77e773f"
server_key_iv: "d5d941dfb63181c60daed5aac53a6a90"
server_key_version: "encryptionkey@0"
I20260926 20:44:48.209409 11513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance:
uuid: "db58290d858d4788ba38b140180080ff"
format_stamp: "Formatted at 2026-09-26 20:44:48 on dist-test-slave-4d5t"
server_key: "c81704b1b2c8d406c5e4699bc77e773f"
server_key_iv: "d5d941dfb63181c60daed5aac53a6a90"
server_key_version: "encryptionkey@0"
I20260926 20:44:48.214883 11513 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260926 20:44:48.219360 11527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:48.221357 11513 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:44:48.221588 11513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/wal
uuid: "db58290d858d4788ba38b140180080ff"
format_stamp: "Formatted at 2026-09-26 20:44:48 on dist-test-slave-4d5t"
server_key: "c81704b1b2c8d406c5e4699bc77e773f"
server_key_iv: "d5d941dfb63181c60daed5aac53a6a90"
server_key_version: "encryptionkey@0"
I20260926 20:44:48.221822 11513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:48.235818 11513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:48.236819 11513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:48.237399 11513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:48.238792 11513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 20:44:48.240656 11513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 20:44:48.240754 11513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:48.240892 11513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 20:44:48.240963 11513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:48.276405 11513 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.195:40875
I20260926 20:44:48.276753 11639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.195:40875 every 8 connection(s)
I20260926 20:44:48.277814 11513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/data/info.pb
I20260926 20:44:48.282068 11378 heartbeater.cc:503] Master 127.23.44.254:43197 was elected leader, sending a full tablet report...
I20260926 20:44:48.284047 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 11513
I20260926 20:44:48.284225 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/wal/instance
I20260926 20:44:48.284832 23731 external_mini_cluster.cc:1428] Setting key e23d2e9b98e2fe2cefce43b1ed545d15
I20260926 20:44:48.292675 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.281377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:42905 (local address 127.23.44.254:43197)
0926 20:44:48.281592 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:48.281597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:48.283291 (+ 1694us) server_negotiation.cc:407] Connection header received
0926 20:44:48.285798 (+ 2507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:48.285803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:48.285891 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:48.286014 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:48.287246 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.288125 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.289770 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.290165 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.290301 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.291797 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:48.291802 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:48.291805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:48.291824 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:48.291867 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:48.291879 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:48.291979 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:48.292205 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 20:44:48.292315 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:44:48.293748 11642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.281907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:43197 (local address 127.23.44.195:42905)
0926 20:44:48.283138 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:48.283189 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:48.285528 (+ 2339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:48.286281 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:48.286293 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:48.286439 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:44:48.287078 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:48.287096 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.288287 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.288292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:48.289610 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:48.289622 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.290394 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.290403 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:48.290646 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.291369 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 20:44:48.291371 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:44:48.291417 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 20:44:48.291497 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:44:48.291980 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:44:48.291993 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:44:48.292004 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 20:44:48.292202 (+ 198us) client_negotiation.cc:240] Negotiation successful
0926 20:44:48.292674 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":375,"threads_started":1}
I20260926 20:44:48.295224 11640 heartbeater.cc:347] Connected to a master server at 127.23.44.254:43197
I20260926 20:44:48.295496 11640 heartbeater.cc:464] Registering TS with master...
I20260926 20:44:48.296213 11640 heartbeater.cc:511] Master 127.23.44.254:43197 requested a full tablet report, sending...
I20260926 20:44:48.297724 11192 ts_manager.cc:194] Registered new tserver with Master: db58290d858d4788ba38b140180080ff (127.23.44.195:40875)
I20260926 20:44:48.298655 11192 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.44.195:42905
I20260926 20:44:48.300746 23731 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 20:44:48.311825 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.305015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44122 (local address 127.23.44.254:43197)
0926 20:44:48.305192 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:48.305197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:48.305694 (+ 497us) server_negotiation.cc:407] Connection header received
0926 20:44:48.305819 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:48.305823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:48.305896 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:48.305988 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:48.306697 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.307730 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.309266 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.309694 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.309794 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.310958 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:48.310963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:48.310967 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:48.310996 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:48.311028 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:48.311040 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:48.311114 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:48.311432 (+ 318us) server_negotiation.cc:299] Negotiation successful
0926 20:44:48.311529 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 20:44:48.317386 11191 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 20:44:48.324947 11191 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 20:44:48.362167 11654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.338303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:40013 (local address 127.0.0.1:51714)
0926 20:44:48.351442 (+ 13139us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:48.351501 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:48.351772 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:48.354234 (+ 2462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:48.354243 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:48.354268 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:48.354750 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:48.354765 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.357025 (+ 2260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.357031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:48.359233 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:48.359248 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.361070 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.361077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:48.361348 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.361402 (+ 54us) client_negotiation.cc:769] Sending connection context
0926 20:44:48.361556 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 20:44:48.361794 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12738,"mutex_wait_us":5,"thread_start_us":14846,"threads_started":1}
I20260926 20:44:48.362186 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.338526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51714 (local address 127.23.44.193:40013)
0926 20:44:48.353236 (+ 14710us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:48.353243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:48.353279 (+ 36us) server_negotiation.cc:407] Connection header received
0926 20:44:48.353395 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:48.353405 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:48.353677 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:48.353918 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:48.354979 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.356830 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.359442 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.360797 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.360943 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.361213 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:48.361603 (+ 390us) server_negotiation.cc:299] Negotiation successful
0926 20:44:48.361836 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14543,"thread_start_us":14410,"threads_started":1}
I20260926 20:44:48.363533 11657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.341976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.23.44.195:40875)
0926 20:44:48.353710 (+ 11734us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:48.353719 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:48.354393 (+ 674us) server_negotiation.cc:407] Connection header received
0926 20:44:48.354500 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:48.354507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:48.354711 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:48.354903 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:48.356851 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.358660 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.361070 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.362478 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.362635 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.362894 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:48.363107 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 20:44:48.363261 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11576,"thread_start_us":11408,"threads_started":1}
I20260926 20:44:48.363688 11655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.338825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59076 (local address 127.23.44.194:33295)
0926 20:44:48.352502 (+ 13677us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:48.352516 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:48.353580 (+ 1064us) server_negotiation.cc:407] Connection header received
0926 20:44:48.353684 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:48.353691 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:48.353908 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:48.354090 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:48.355079 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.356100 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.358606 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.360231 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.360378 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.361449 (+ 1071us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:48.361825 (+ 376us) server_negotiation.cc:299] Negotiation successful
0926 20:44:48.361998 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13442,"thread_start_us":13701,"threads_started":1}
I20260926 20:44:48.362814 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.338564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:33295 (local address 127.0.0.1:59076)
0926 20:44:48.352905 (+ 14341us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:48.352928 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:48.353074 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:48.354389 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:48.354395 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:48.354427 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:48.354890 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:48.354902 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.356241 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.356247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:48.358418 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:48.358430 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.361337 (+ 2907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.361345 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:48.361520 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.361557 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 20:44:48.361630 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 20:44:48.362442 (+ 812us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14186,"mutex_wait_us":666,"thread_start_us":12258,"threads_started":1}
I20260926 20:44:48.364369 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.341164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.195:40875 (local address 127.0.0.1:39022)
0926 20:44:48.354129 (+ 12965us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:48.354148 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:48.354258 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:48.355166 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:48.355170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:48.355200 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:48.355820 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:48.355856 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.358814 (+ 2958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.358820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:48.360849 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:48.360861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.362678 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.362685 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:48.362904 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.362940 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 20:44:48.363059 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 20:44:48.363187 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12846,"thread_start_us":13560,"threads_started":1}
I20260926 20:44:48.366322 11313 tablet_service.cc:1467] Processing CreateTablet for tablet 8dec737db0f54ac4a43478eeec4a109e (DEFAULT_TABLE table=test-table [id=6ae922ecc3874ef3810c2e4a54e93c08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:48.367535 11313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dec737db0f54ac4a43478eeec4a109e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:48.367295 11444 tablet_service.cc:1467] Processing CreateTablet for tablet 8dec737db0f54ac4a43478eeec4a109e (DEFAULT_TABLE table=test-table [id=6ae922ecc3874ef3810c2e4a54e93c08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:48.367662 11575 tablet_service.cc:1467] Processing CreateTablet for tablet 8dec737db0f54ac4a43478eeec4a109e (DEFAULT_TABLE table=test-table [id=6ae922ecc3874ef3810c2e4a54e93c08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 20:44:48.368677 11444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dec737db0f54ac4a43478eeec4a109e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:48.369050 11575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dec737db0f54ac4a43478eeec4a109e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:48.379340 11658 tablet_bootstrap.cc:492] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50: Bootstrap starting.
I20260926 20:44:48.383389 11658 tablet_bootstrap.cc:654] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:48.384452 11658 log.cc:824] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:48.385633 11659 tablet_bootstrap.cc:492] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e: Bootstrap starting.
I20260926 20:44:48.387396 11660 tablet_bootstrap.cc:492] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff: Bootstrap starting.
I20260926 20:44:48.390651 11660 tablet_bootstrap.cc:654] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:48.391021 11659 tablet_bootstrap.cc:654] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:48.391815 11660 log.cc:824] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:48.392530 11659 log.cc:824] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:48.395107 11659 tablet_bootstrap.cc:492] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e: No bootstrap required, opened a new log
I20260926 20:44:48.395191 11658 tablet_bootstrap.cc:492] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50: No bootstrap required, opened a new log
I20260926 20:44:48.395417 11659 ts_tablet_manager.cc:1397] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260926 20:44:48.395447 11660 tablet_bootstrap.cc:492] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff: No bootstrap required, opened a new log
I20260926 20:44:48.395488 11658 ts_tablet_manager.cc:1397] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.000s
I20260926 20:44:48.395684 11660 ts_tablet_manager.cc:1397] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260926 20:44:48.403754 11659 raft_consensus.cc:348] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f630cec46f6490c983e1771976c0c50" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40013 } } peers { permanent_uuid: "59e6550dcaf54b41a6ba97c140c03f8e" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 33295 } } peers { permanent_uuid: "db58290d858d4788ba38b140180080ff" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40875 } }
I20260926 20:44:48.404186 11659 raft_consensus.cc:374] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:48.404279 11659 raft_consensus.cc:729] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59e6550dcaf54b41a6ba97c140c03f8e, State: Initialized, Role: FOLLOWER
I20260926 20:44:48.404958 11659 consensus_queue.cc:260] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f630cec46f6490c983e1771976c0c50" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40013 } } peers { permanent_uuid: "59e6550dcaf54b41a6ba97c140c03f8e" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 33295 } } peers { permanent_uuid: "db58290d858d4788ba38b140180080ff" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40875 } }
I20260926 20:44:48.406081 11509 heartbeater.cc:503] Master 127.23.44.254:43197 was elected leader, sending a full tablet report...
I20260926 20:44:48.406569 11658 raft_consensus.cc:348] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f630cec46f6490c983e1771976c0c50" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40013 } } peers { permanent_uuid: "59e6550dcaf54b41a6ba97c140c03f8e" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 33295 } } peers { permanent_uuid: "db58290d858d4788ba38b140180080ff" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40875 } }
I20260926 20:44:48.406569 11660 raft_consensus.cc:348] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f630cec46f6490c983e1771976c0c50" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40013 } } peers { permanent_uuid: "59e6550dcaf54b41a6ba97c140c03f8e" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 33295 } } peers { permanent_uuid: "db58290d858d4788ba38b140180080ff" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40875 } }
I20260926 20:44:48.407019 11660 raft_consensus.cc:374] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:48.407019 11658 raft_consensus.cc:374] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:48.407130 11660 raft_consensus.cc:729] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db58290d858d4788ba38b140180080ff, State: Initialized, Role: FOLLOWER
I20260926 20:44:48.407123 11658 raft_consensus.cc:729] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f630cec46f6490c983e1771976c0c50, State: Initialized, Role: FOLLOWER
I20260926 20:44:48.407742 11658 consensus_queue.cc:260] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f630cec46f6490c983e1771976c0c50" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40013 } } peers { permanent_uuid: "59e6550dcaf54b41a6ba97c140c03f8e" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 33295 } } peers { permanent_uuid: "db58290d858d4788ba38b140180080ff" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40875 } }
I20260926 20:44:48.407784 11660 consensus_queue.cc:260] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f630cec46f6490c983e1771976c0c50" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40013 } } peers { permanent_uuid: "59e6550dcaf54b41a6ba97c140c03f8e" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 33295 } } peers { permanent_uuid: "db58290d858d4788ba38b140180080ff" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40875 } }
I20260926 20:44:48.408416 11659 ts_tablet_manager.cc:1428] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
I20260926 20:44:48.409776 11640 heartbeater.cc:503] Master 127.23.44.254:43197 was elected leader, sending a full tablet report...
I20260926 20:44:48.410082 11658 ts_tablet_manager.cc:1428] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50: Time spent starting tablet: real 0.014s user 0.009s sys 0.007s
I20260926 20:44:48.410359 11660 ts_tablet_manager.cc:1428] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff: Time spent starting tablet: real 0.015s user 0.014s sys 0.002s
W20260926 20:44:48.503693 11379 tablet.cc:2369] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:44:48.530341 11510 tablet.cc:2369] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 20:44:48.530355 11641 tablet.cc:2369] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:44:48.601658 11665 raft_consensus.cc:482] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 20:44:48.601979 11665 raft_consensus.cc:504] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f630cec46f6490c983e1771976c0c50" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40013 } } peers { permanent_uuid: "59e6550dcaf54b41a6ba97c140c03f8e" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 33295 } } peers { permanent_uuid: "db58290d858d4788ba38b140180080ff" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40875 } }
I20260926 20:44:48.603821 11665 leader_election.cc:288] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f630cec46f6490c983e1771976c0c50 (127.23.44.193:40013), 59e6550dcaf54b41a6ba97c140c03f8e (127.23.44.194:33295)
I20260926 20:44:48.610116 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.604515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:51437 (local address 127.23.44.193:40013)
0926 20:44:48.604733 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:48.604739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:48.604757 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:44:48.604831 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:48.604838 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:48.604925 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:48.605022 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:48.605711 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.606709 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.608331 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.609173 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.609353 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.609426 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:48.609717 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 20:44:48.609842 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 20:44:48.610167 11642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.604044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.193:40013 (local address 127.23.44.195:51437)
0926 20:44:48.604286 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:48.604307 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:48.604435 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:48.605262 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:48.605269 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:48.605303 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:48.605525 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:48.605541 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.606853 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.606865 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:48.608156 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:48.608169 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.609340 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.609346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:48.609552 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.609580 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 20:44:48.609689 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 20:44:48.609833 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260926 20:44:48.611213 11333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dec737db0f54ac4a43478eeec4a109e" candidate_uuid: "db58290d858d4788ba38b140180080ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f630cec46f6490c983e1771976c0c50" is_pre_election: true
I20260926 20:44:48.611842 11333 raft_consensus.cc:2454] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db58290d858d4788ba38b140180080ff in term 0.
I20260926 20:44:48.612663 11655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.604762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.44.195:49387 (local address 127.23.44.194:33295)
0926 20:44:48.604945 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:48.604951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:48.605609 (+ 658us) server_negotiation.cc:407] Connection header received
0926 20:44:48.605774 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:48.605780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:48.605879 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:48.605979 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 20:44:48.606572 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.607642 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.610195 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.611428 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.611577 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.611643 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:48.612214 (+ 571us) server_negotiation.cc:299] Negotiation successful
0926 20:44:48.612370 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 20:44:48.612756 11531 leader_election.cc:302] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f630cec46f6490c983e1771976c0c50, db58290d858d4788ba38b140180080ff; no voters:
I20260926 20:44:48.613181 11665 raft_consensus.cc:2781] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 20:44:48.613399 11665 raft_consensus.cc:482] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 20:44:48.613536 11667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.604087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.194:33295 (local address 127.23.44.195:49387)
0926 20:44:48.605493 (+ 1406us) negotiation.cc:107] Waiting for socket to connect
0926 20:44:48.605510 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:44:48.605636 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:44:48.606032 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:44:48.606037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:44:48.606064 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 20:44:48.606408 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:48.606420 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.607833 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.607839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:48.609976 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:44:48.609990 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.611643 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.611648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:44:48.611966 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.612019 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 20:44:48.612937 (+ 918us) client_negotiation.cc:240] Negotiation successful
0926 20:44:48.613096 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1265,"thread_start_us":1028,"threads_started":1}
I20260926 20:44:48.613533 11665 raft_consensus.cc:3037] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:48.614658 11464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dec737db0f54ac4a43478eeec4a109e" candidate_uuid: "db58290d858d4788ba38b140180080ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e6550dcaf54b41a6ba97c140c03f8e" is_pre_election: true
I20260926 20:44:48.615187 11464 raft_consensus.cc:2454] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db58290d858d4788ba38b140180080ff in term 0.
I20260926 20:44:48.616349 11665 raft_consensus.cc:504] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f630cec46f6490c983e1771976c0c50" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40013 } } peers { permanent_uuid: "59e6550dcaf54b41a6ba97c140c03f8e" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 33295 } } peers { permanent_uuid: "db58290d858d4788ba38b140180080ff" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40875 } }
I20260926 20:44:48.617131 11665 leader_election.cc:288] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [CANDIDATE]: Term 1 election: Requested vote from peers 9f630cec46f6490c983e1771976c0c50 (127.23.44.193:40013), 59e6550dcaf54b41a6ba97c140c03f8e (127.23.44.194:33295)
I20260926 20:44:48.617638 11333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dec737db0f54ac4a43478eeec4a109e" candidate_uuid: "db58290d858d4788ba38b140180080ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f630cec46f6490c983e1771976c0c50"
I20260926 20:44:48.617662 11464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dec737db0f54ac4a43478eeec4a109e" candidate_uuid: "db58290d858d4788ba38b140180080ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59e6550dcaf54b41a6ba97c140c03f8e"
I20260926 20:44:48.617866 11464 raft_consensus.cc:3037] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:48.617854 11333 raft_consensus.cc:3037] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:48.620270 11464 raft_consensus.cc:2454] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db58290d858d4788ba38b140180080ff in term 1.
I20260926 20:44:48.620291 11333 raft_consensus.cc:2454] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db58290d858d4788ba38b140180080ff in term 1.
I20260926 20:44:48.620832 11528 leader_election.cc:302] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59e6550dcaf54b41a6ba97c140c03f8e, db58290d858d4788ba38b140180080ff; no voters:
I20260926 20:44:48.621286 11665 raft_consensus.cc:2781] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:48.621802 11665 raft_consensus.cc:686] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [term 1 LEADER]: Becoming Leader. State: Replica: db58290d858d4788ba38b140180080ff, State: Running, Role: LEADER
I20260926 20:44:48.622287 11665 consensus_queue.cc:237] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f630cec46f6490c983e1771976c0c50" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40013 } } peers { permanent_uuid: "59e6550dcaf54b41a6ba97c140c03f8e" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 33295 } } peers { permanent_uuid: "db58290d858d4788ba38b140180080ff" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40875 } }
I20260926 20:44:48.626925 11192 catalog_manager.cc:5495] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff reported cstate change: term changed from 0 to 1, leader changed from <none> to db58290d858d4788ba38b140180080ff (127.23.44.195). New cstate: current_term: 1 leader_uuid: "db58290d858d4788ba38b140180080ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f630cec46f6490c983e1771976c0c50" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59e6550dcaf54b41a6ba97c140c03f8e" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 33295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db58290d858d4788ba38b140180080ff" member_type: VOTER last_known_addr { host: "127.23.44.195" port: 40875 } health_report { overall_health: HEALTHY } } }
I20260926 20:44:48.670240 11657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.664479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39032 (local address 127.23.44.195:40875)
0926 20:44:48.664743 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:48.664751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:48.664807 (+ 56us) server_negotiation.cc:407] Connection header received
0926 20:44:48.665003 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:48.665008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:48.665103 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:48.665230 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:44:48.665629 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.666456 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.667851 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.668311 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.668424 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.669019 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:44:48.669553 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:44:48.669670 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:48.669834 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 20:44:48.669953 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260926 20:44:48.680395 11464 raft_consensus.cc:1261] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e [term 1 FOLLOWER]: Refusing update from remote peer db58290d858d4788ba38b140180080ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:44:48.680480 11333 raft_consensus.cc:1261] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50 [term 1 FOLLOWER]: Refusing update from remote peer db58290d858d4788ba38b140180080ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:44:48.681604 11668 consensus_queue.cc:1035] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f630cec46f6490c983e1771976c0c50" member_type: VOTER last_known_addr { host: "127.23.44.193" port: 40013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:44:48.682041 11665 consensus_queue.cc:1035] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "59e6550dcaf54b41a6ba97c140c03f8e" member_type: VOTER last_known_addr { host: "127.23.44.194" port: 33295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:44:48.693534 11671 mvcc.cc:205] Tried to move back new op lower bound from 7333705681622962176 to 7333705681400524800. Current Snapshot: MvccSnapshot[applied={T|T < 7333705681622962176}]
I20260926 20:44:48.694080 11673 mvcc.cc:205] Tried to move back new op lower bound from 7333705681622962176 to 7333705681400524800. Current Snapshot: MvccSnapshot[applied={T|T < 7333705681622962176}]
I20260926 20:44:48.712129 11192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:44:48.712432 11192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 20:44:48.716358 11192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b27646ca790411bbbe5ea6b0c962e6e: Sending DeleteTablet for 3 replicas of tablet 8dec737db0f54ac4a43478eeec4a109e
I20260926 20:44:48.717785 11444 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dec737db0f54ac4a43478eeec4a109e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:48 UTC) from {username='slave'} at 127.0.0.1:59076
I20260926 20:44:48.717794 11313 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dec737db0f54ac4a43478eeec4a109e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:48 UTC) from {username='slave'} at 127.0.0.1:51714
I20260926 20:44:48.718580 11575 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dec737db0f54ac4a43478eeec4a109e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 20:44:48 UTC) from {username='slave'} at 127.0.0.1:39022
I20260926 20:44:48.718978 11681 tablet_replica.cc:330] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50: stopping tablet replica
I20260926 20:44:48.719178 11680 tablet_replica.cc:330] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e: stopping tablet replica
I20260926 20:44:48.719530 11680 raft_consensus.cc:2229] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:44:48.719488 11681 raft_consensus.cc:2229] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 20:44:48.720088 11681 raft_consensus.cc:2258] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:48.720086 11680 raft_consensus.cc:2258] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:48.720187 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 11251
I20260926 20:44:48.721065 11682 tablet_replica.cc:330] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff: stopping tablet replica
I20260926 20:44:48.721567 11682 raft_consensus.cc:2229] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [term 1 LEADER]: Raft consensus shutting down.
I20260926 20:44:48.722317 11682 raft_consensus.cc:2258] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 20:44:48.723053 11680 ts_tablet_manager.cc:1905] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:48.723958 11681 ts_tablet_manager.cc:1905] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:48.732496 11680 ts_tablet_manager.cc:1918] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:48.732725 11680 log.cc:1196] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-1/wal/wals/8dec737db0f54ac4a43478eeec4a109e
I20260926 20:44:48.733194 11681 ts_tablet_manager.cc:1918] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:48.733338 11680 ts_tablet_manager.cc:1939] T 8dec737db0f54ac4a43478eeec4a109e P 59e6550dcaf54b41a6ba97c140c03f8e: Deleting consensus metadata
I20260926 20:44:48.733388 11681 log.cc:1196] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-0/wal/wals/8dec737db0f54ac4a43478eeec4a109e
I20260926 20:44:48.733958 11681 ts_tablet_manager.cc:1939] T 8dec737db0f54ac4a43478eeec4a109e P 9f630cec46f6490c983e1771976c0c50: Deleting consensus metadata
I20260926 20:44:48.735855 11176 catalog_manager.cc:4841] TS 59e6550dcaf54b41a6ba97c140c03f8e (127.23.44.194:33295): tablet 8dec737db0f54ac4a43478eeec4a109e (table test-table [id=6ae922ecc3874ef3810c2e4a54e93c08]) successfully deleted
I20260926 20:44:48.736588 11682 ts_tablet_manager.cc:1905] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 20:44:48.740046 11179 catalog_manager.cc:4841] TS 9f630cec46f6490c983e1771976c0c50 (127.23.44.193:40013): tablet 8dec737db0f54ac4a43478eeec4a109e (table test-table [id=6ae922ecc3874ef3810c2e4a54e93c08]) successfully deleted
I20260926 20:44:48.742774 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.722456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51716 (local address 127.23.44.193:40013)
0926 20:44:48.722638 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:48.722645 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:48.733600 (+ 10955us) server_negotiation.cc:407] Connection header received
0926 20:44:48.733905 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:48.733911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:48.734029 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:48.734117 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:48.734858 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.736035 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.738202 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.738865 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.738997 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.740779 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:48.740790 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:48.740809 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:48.741708 (+ 899us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:48.741778 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:48.741806 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:48.741989 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:48.742185 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 20:44:48.742317 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 20:44:48.743808 11373 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:48.746558 11682 ts_tablet_manager.cc:1918] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 20:44:48.746989 11682 log.cc:1196] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff: Deleting WAL directory at /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790455151607908-23731-0/minicluster-data/ts-2/wal/wals/8dec737db0f54ac4a43478eeec4a109e
I20260926 20:44:48.747848 11682 ts_tablet_manager.cc:1939] T 8dec737db0f54ac4a43478eeec4a109e P db58290d858d4788ba38b140180080ff: Deleting consensus metadata
I20260926 20:44:48.749769 11177 catalog_manager.cc:4841] TS db58290d858d4788ba38b140180080ff (127.23.44.195:40875): tablet 8dec737db0f54ac4a43478eeec4a109e (table test-table [id=6ae922ecc3874ef3810c2e4a54e93c08]) successfully deleted
I20260926 20:44:48.882110 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 11251
I20260926 20:44:48.903376 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 11382
I20260926 20:44:48.911902 11655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:48.904281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59078 (local address 127.23.44.194:33295)
0926 20:44:48.904467 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:48.904471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:48.904492 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:44:48.904685 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:48.904690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:48.904780 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:48.904886 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:48.905371 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.906259 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.908173 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:48.908839 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:48.909032 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:48.910311 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:48.910320 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:48.910335 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:48.911105 (+ 770us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:48.911163 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:48.911184 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:48.911281 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:48.911443 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 20:44:48.911540 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:44:48.912807 11504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:49.004215 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 11382
I20260926 20:44:49.025990 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 11513
I20260926 20:44:49.034964 11657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:49.027341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39038 (local address 127.23.44.195:40875)
0926 20:44:49.027624 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:49.027632 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:49.027655 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:44:49.027732 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:49.027738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:49.027843 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:49.027980 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:49.028354 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:49.029633 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:49.030895 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:49.031637 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:49.031775 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:49.033230 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:49.033241 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:49.033262 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:49.034064 (+ 802us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:49.034135 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:49.034165 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:49.034277 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:49.034462 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 20:44:49.034572 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260926 20:44:49.036095 11635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:49.167167 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 11513
I20260926 20:44:49.187019 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 11161
I20260926 20:44:49.188345 11222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:49.297899 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 11161
2026-09-26T20:44:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3240 ms)
[----------] 6 tests from EncryptionPolicyTest (23954 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:44:49 dist-test-slave-4d5t krb5kdc[11690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:44:49 dist-test-slave-4d5t krb5kdc[11690](info): set up 2 sockets
Sep 26 20:44:49 dist-test-slave-4d5t krb5kdc[11690](info): commencing operation
krb5kdc: starting...
W20260926 20:44:50.522034 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:44:50 dist-test-slave-4d5t krb5kdc[11690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:44:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:44:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:44:50.813730 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40617
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:51.178786 11703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:51.179183 11703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:51.179363 11703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:51.187755 11703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:44:51.187963 11703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:51.188058 11703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:51.188151 11703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:44:51.188241 11703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:44:51.198755 11703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40617
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:51.200212 11703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:51.201642 11703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:51.214632 11711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:51.214749 11709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:51.214771 11708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:51.215873 11703 server_base.cc:1034] running on GCE node
I20260926 20:44:51.217064 11703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:51.218822 11703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:51.220273 11703 hybrid_clock.cc:648] HybridClock initialized: now 1790455491220140 us; error 66 us; skew 500 ppm
I20260926 20:44:51.221012 11703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:51 dist-test-slave-4d5t krb5kdc[11690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455491, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:51.228834 11703 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:44:51.230670 11703 webserver.cc:460] Webserver started at http://127.0.0.1:37357/ using document root <none> and password file <none>
I20260926 20:44:51.231467 11703 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:51.231616 11703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:51.231979 11703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:51.235461 11703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "e73424e850ed4f33b23a6ffbd818bdea"
format_stamp: "Formatted at 2026-09-26 20:44:51 on dist-test-slave-4d5t"
server_key: "00db3070b16604de5b0b462e7baaddd9"
server_key_iv: "cab0155f610737285a3e264395668cf6"
server_key_version: "encryptionkey@0"
I20260926 20:44:51.236331 11703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "e73424e850ed4f33b23a6ffbd818bdea"
format_stamp: "Formatted at 2026-09-26 20:44:51 on dist-test-slave-4d5t"
server_key: "00db3070b16604de5b0b462e7baaddd9"
server_key_iv: "cab0155f610737285a3e264395668cf6"
server_key_version: "encryptionkey@0"
I20260926 20:44:51.241998 11703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 20:44:51.245688 11718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:51.247189 11703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:44:51.247413 11703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "e73424e850ed4f33b23a6ffbd818bdea"
format_stamp: "Formatted at 2026-09-26 20:44:51 on dist-test-slave-4d5t"
server_key: "00db3070b16604de5b0b462e7baaddd9"
server_key_iv: "cab0155f610737285a3e264395668cf6"
server_key_version: "encryptionkey@0"
I20260926 20:44:51.247640 11703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:51.265147 11703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:51.270725 11703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:51.271431 11703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:51.294359 11703 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40617
I20260926 20:44:51.294479 11769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40617 every 8 connection(s)
I20260926 20:44:51.295902 11703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:44:51.300006 11770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:51.305104 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 11703
I20260926 20:44:51.309912 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:44:51.310616 23731 external_mini_cluster.cc:1428] Setting key 2af11a5a9b4c2ef471216c045180f7f3
I20260926 20:44:51.311045 11770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea: Bootstrap starting.
I20260926 20:44:51.315550 11770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:51.317126 11770 log.cc:824] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:51.320642 11770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea: No bootstrap required, opened a new log
Sep 26 20:44:51 dist-test-slave-4d5t krb5kdc[11690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 20:44:51.330884 11770 raft_consensus.cc:348] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e73424e850ed4f33b23a6ffbd818bdea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40617 } }
I20260926 20:44:51.331511 11770 raft_consensus.cc:374] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:51.331652 11770 raft_consensus.cc:729] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e73424e850ed4f33b23a6ffbd818bdea, State: Initialized, Role: FOLLOWER
I20260926 20:44:51.332422 11770 consensus_queue.cc:260] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e73424e850ed4f33b23a6ffbd818bdea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40617 } }
I20260926 20:44:51.332787 11770 raft_consensus.cc:388] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:44:51.332967 11770 raft_consensus.cc:482] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:44:51.333182 11770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:51.336511 11770 raft_consensus.cc:504] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e73424e850ed4f33b23a6ffbd818bdea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40617 } }
I20260926 20:44:51.337461 11770 leader_election.cc:302] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e73424e850ed4f33b23a6ffbd818bdea; no voters:
I20260926 20:44:51.338583 11770 leader_election.cc:288] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:44:51.338722 11775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:51.339358 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:51.312862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52285 (local address 127.0.0.1:40617)
0926 20:44:51.314101 (+ 1239us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:51.314118 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:51.314173 (+ 55us) server_negotiation.cc:407] Connection header received
0926 20:44:51.317022 (+ 2849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:51.317053 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:51.317592 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:51.318104 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:51.319717 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:51.321846 (+ 2129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:51.323304 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:51.324539 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:51.324804 (+ 265us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:51.329168 (+ 4364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:51.329182 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:51.329207 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:51.329284 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:51.333744 (+ 4460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:51.334747 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:51.334756 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:51.334766 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:51.334997 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:51.335548 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:51.335553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:51.335557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:51.336186 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:51.336649 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:51.337398 (+ 749us) server_negotiation.cc:299] Negotiation successful
0926 20:44:51.337965 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":939,"thread_start_us":619,"threads_started":1}
I20260926 20:44:51.341298 11775 raft_consensus.cc:686] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [term 1 LEADER]: Becoming Leader. State: Replica: e73424e850ed4f33b23a6ffbd818bdea, State: Running, Role: LEADER
I20260926 20:44:51.341740 11770 sys_catalog.cc:564] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:44:51.342108 11775 consensus_queue.cc:237] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e73424e850ed4f33b23a6ffbd818bdea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40617 } }
I20260926 20:44:51.344347 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:44:51.352813 11777 sys_catalog.cc:455] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [sys.catalog]: SysCatalogTable state changed. Reason: New leader e73424e850ed4f33b23a6ffbd818bdea. Latest consensus state: current_term: 1 leader_uuid: "e73424e850ed4f33b23a6ffbd818bdea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e73424e850ed4f33b23a6ffbd818bdea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40617 } } }
I20260926 20:44:51.353474 11777 sys_catalog.cc:458] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:51.352463 11776 sys_catalog.cc:455] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e73424e850ed4f33b23a6ffbd818bdea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e73424e850ed4f33b23a6ffbd818bdea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40617 } } }
I20260926 20:44:51.354660 11776 sys_catalog.cc:458] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:51.356391 11792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:44:51.370924 11792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:44:51.372764 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:51.350402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35279 (local address 127.0.0.1:40617)
0926 20:44:51.350626 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:51.350634 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:51.350660 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:44:51.350713 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:51.350718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:51.350781 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:51.350921 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:51.356739 (+ 5818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:51.357550 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:51.358864 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:51.359338 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:51.359476 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:51.362673 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:51.362680 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:51.362685 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:51.362735 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:51.368405 (+ 5670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:51.369690 (+ 1285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:51.369699 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:51.369703 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:51.370063 (+ 360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:51.370755 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:51.370762 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:51.370767 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:51.371275 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:51.371593 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:51.372118 (+ 525us) server_negotiation.cc:299] Negotiation successful
0926 20:44:51.372339 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 20:44:51.381399 11792 catalog_manager.cc:1347] Generated new cluster ID: f2222ed077fd45b88391a86a0a8c9bdb
I20260926 20:44:51.381551 11792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:44:51.415710 11792 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:44:51.417171 11792 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:44:51.432515 11792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e73424e850ed4f33b23a6ffbd818bdea: Generated new TSK 0
I20260926 20:44:51.433503 11792 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:44:51.453768 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 11703
I20260926 20:44:51.455194 11765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:51.555848 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 11703
W20260926 20:44:51.573553 11688 connection.cc:535] client connection to 127.0.0.1:40617 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40617): Connection reset by peer (error 104)
2026-09-26T20:44:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2251 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:44:51 dist-test-slave-4d5t krb5kdc[11802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:44:51 dist-test-slave-4d5t krb5kdc[11802](info): set up 2 sockets
Sep 26 20:44:51 dist-test-slave-4d5t krb5kdc[11802](info): commencing operation
krb5kdc: starting...
W20260926 20:44:52.725857 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:44:52 dist-test-slave-4d5t krb5kdc[11802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:44:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:44:53.004120 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35447
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:53.371030 11815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:53.371413 11815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:53.371592 11815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:53.379196 11815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:44:53.379374 11815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:53.379495 11815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:53.379601 11815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:44:53.379685 11815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:44:53.391328 11815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35447
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:53.393064 11815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:53.394568 11815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:53.407012 11820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:53.407040 11823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:53.407250 11821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:53.408717 11815 server_base.cc:1034] running on GCE node
I20260926 20:44:53.409543 11815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:53.411052 11815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:53.412285 11815 hybrid_clock.cc:648] HybridClock initialized: now 1790455493412223 us; error 64 us; skew 500 ppm
I20260926 20:44:53.412734 11815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:53 dist-test-slave-4d5t krb5kdc[11802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455493, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:53.416024 11815 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:44:53.417748 11815 webserver.cc:460] Webserver started at http://127.0.0.1:41511/ using document root <none> and password file <none>
I20260926 20:44:53.418473 11815 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:53.418634 11815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:53.418959 11815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:53.422497 11815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "5995269959144d11bf0f6bbacf7ecad5"
format_stamp: "Formatted at 2026-09-26 20:44:53 on dist-test-slave-4d5t"
server_key: "87caa1e8760c11a26af53d8712221de9"
server_key_iv: "a574fecd8025ecbc4177b255209b7366"
server_key_version: "encryptionkey@0"
I20260926 20:44:53.423448 11815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "5995269959144d11bf0f6bbacf7ecad5"
format_stamp: "Formatted at 2026-09-26 20:44:53 on dist-test-slave-4d5t"
server_key: "87caa1e8760c11a26af53d8712221de9"
server_key_iv: "a574fecd8025ecbc4177b255209b7366"
server_key_version: "encryptionkey@0"
I20260926 20:44:53.429698 11815 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260926 20:44:53.433691 11830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:53.435283 11815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 20:44:53.435508 11815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "5995269959144d11bf0f6bbacf7ecad5"
format_stamp: "Formatted at 2026-09-26 20:44:53 on dist-test-slave-4d5t"
server_key: "87caa1e8760c11a26af53d8712221de9"
server_key_iv: "a574fecd8025ecbc4177b255209b7366"
server_key_version: "encryptionkey@0"
I20260926 20:44:53.435712 11815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:53.465919 11815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:53.471028 11815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:53.471386 11815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:53.493752 11815 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35447
I20260926 20:44:53.493785 11881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35447 every 8 connection(s)
I20260926 20:44:53.495199 11815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:44:53.499056 11882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:53.502094 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 11815
I20260926 20:44:53.502307 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:44:53.502987 23731 external_mini_cluster.cc:1428] Setting key ade08bc25c263b8840df17ad380837c3
I20260926 20:44:53.513692 11882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5: Bootstrap starting.
I20260926 20:44:53.518826 11882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:53.520571 11882 log.cc:824] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:53.521251 11885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:53.505929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33051 (local address 127.0.0.1:35447)
0926 20:44:53.506785 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:53.506801 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:53.506856 (+ 55us) server_negotiation.cc:407] Connection header received
0926 20:44:53.508917 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:53.508950 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:53.509581 (+ 631us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:53.510061 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:53.510660 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:53.512718 (+ 2058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:53.514414 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:53.516083 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:53.516334 (+ 251us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:53.517756 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:53.517770 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:53.517793 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:53.518599 (+ 806us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:53.518730 (+ 131us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:53.518767 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:53.518965 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:53.519389 (+ 424us) server_negotiation.cc:299] Negotiation successful
0926 20:44:53.519639 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":381,"threads_started":1}
I20260926 20:44:53.524116 11882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5: No bootstrap required, opened a new log
I20260926 20:44:53.526444 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:44:53.537061 11882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5995269959144d11bf0f6bbacf7ecad5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35447 } }
I20260926 20:44:53.537902 11882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:53.538100 11882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5995269959144d11bf0f6bbacf7ecad5, State: Initialized, Role: FOLLOWER
I20260926 20:44:53.538966 11882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5995269959144d11bf0f6bbacf7ecad5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35447 } }
I20260926 20:44:53.539359 11882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:44:53.539563 11882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:44:53.539824 11882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:53.540869 11885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:53.532512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53291 (local address 127.0.0.1:35447)
0926 20:44:53.532773 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:53.532780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:53.533058 (+ 278us) server_negotiation.cc:407] Connection header received
0926 20:44:53.533313 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:53.533321 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:53.533460 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:53.533721 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:53.534401 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:53.535715 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:53.537374 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:53.538356 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:53.538545 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:53.539926 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:53.539935 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:53.539942 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:53.539967 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:53.540053 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:53.540076 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:53.540223 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:53.540430 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 20:44:53.540491 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260926 20:44:53.543426 11882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5995269959144d11bf0f6bbacf7ecad5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35447 } }
I20260926 20:44:53.544061 11882 leader_election.cc:302] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5995269959144d11bf0f6bbacf7ecad5; no voters:
I20260926 20:44:53.544775 11882 leader_election.cc:288] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:44:53.544939 11892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:53.546383 11892 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [term 1 LEADER]: Becoming Leader. State: Replica: 5995269959144d11bf0f6bbacf7ecad5, State: Running, Role: LEADER
I20260926 20:44:53.546938 11892 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5995269959144d11bf0f6bbacf7ecad5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35447 } }
I20260926 20:44:53.547369 11882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:44:53.550036 11893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5995269959144d11bf0f6bbacf7ecad5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5995269959144d11bf0f6bbacf7ecad5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35447 } } }
I20260926 20:44:53.550480 11893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:53.550134 11894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5995269959144d11bf0f6bbacf7ecad5. Latest consensus state: current_term: 1 leader_uuid: "5995269959144d11bf0f6bbacf7ecad5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5995269959144d11bf0f6bbacf7ecad5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35447 } } }
I20260926 20:44:53.550603 11894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5 [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:53.554394 11898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:44:53.560168 11898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:44:53.572285 11898 catalog_manager.cc:1347] Generated new cluster ID: 84bfe41e3bff496fb0c0f95e3475e7e4
I20260926 20:44:53.572465 11898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:44:53.597071 11898 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:44:53.598466 11898 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:44:53.609695 11898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5995269959144d11bf0f6bbacf7ecad5: Generated new TSK 0
I20260926 20:44:53.610635 11898 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:44:53.619900 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 11815
W20260926 20:44:53.621299 11877 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33051
W20260926 20:44:53.622226 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 11815: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:44:53.622435 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 11815
W20260926 20:44:53.640354 11800 connection.cc:535] client connection to 127.0.0.1:35447 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35447): Connection reset by peer (error 104)
2026-09-26T20:44:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2066 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:44:53 dist-test-slave-4d5t krb5kdc[11913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:44:53 dist-test-slave-4d5t krb5kdc[11913](info): set up 2 sockets
Sep 26 20:44:53 dist-test-slave-4d5t krb5kdc[11913](info): commencing operation
krb5kdc: starting...
W20260926 20:44:54.821648 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:44:55 dist-test-slave-4d5t krb5kdc[11913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:44:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:44:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:44:55.130115 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39775
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:55.471454 11926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:55.471829 11926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:55.472010 11926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:55.479395 11926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:44:55.479585 11926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:55.479699 11926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:55.479811 11926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:44:55.479898 11926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:44:55.490056 11926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39775
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:55.491458 11926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:55.492817 11926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:55.505120 11931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:55.505168 11934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:55.505084 11932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:55.506003 11926 server_base.cc:1034] running on GCE node
I20260926 20:44:55.506812 11926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:55.508481 11926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:55.509755 11926 hybrid_clock.cc:648] HybridClock initialized: now 1790455495509715 us; error 33 us; skew 500 ppm
I20260926 20:44:55.510141 11926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:55 dist-test-slave-4d5t krb5kdc[11913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455495, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:55.514210 11926 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:44:55.515890 11926 webserver.cc:460] Webserver started at http://127.0.0.1:39873/ using document root <none> and password file <none>
I20260926 20:44:55.516546 11926 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:55.516727 11926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:55.517001 11926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:55.520057 11926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "d2a52b52da39407a92bdb0fc0266bfdf"
format_stamp: "Formatted at 2026-09-26 20:44:55 on dist-test-slave-4d5t"
server_key: "733e2ae1d640aa08936ceda97c05f8f4"
server_key_iv: "309526598525bbf7f2fcd2b0bc134abe"
server_key_version: "encryptionkey@0"
I20260926 20:44:55.520895 11926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "d2a52b52da39407a92bdb0fc0266bfdf"
format_stamp: "Formatted at 2026-09-26 20:44:55 on dist-test-slave-4d5t"
server_key: "733e2ae1d640aa08936ceda97c05f8f4"
server_key_iv: "309526598525bbf7f2fcd2b0bc134abe"
server_key_version: "encryptionkey@0"
I20260926 20:44:55.526490 11926 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:44:55.530447 11941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:55.532290 11926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 20:44:55.532516 11926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "d2a52b52da39407a92bdb0fc0266bfdf"
format_stamp: "Formatted at 2026-09-26 20:44:55 on dist-test-slave-4d5t"
server_key: "733e2ae1d640aa08936ceda97c05f8f4"
server_key_iv: "309526598525bbf7f2fcd2b0bc134abe"
server_key_version: "encryptionkey@0"
I20260926 20:44:55.532835 11926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:55.539093 11926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:55.543888 11926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:55.544226 11926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:55.566147 11926 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39775
I20260926 20:44:55.566191 11992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39775 every 8 connection(s)
I20260926 20:44:55.568037 11926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:44:55.572854 11993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:55.577186 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 11926
I20260926 20:44:55.577396 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:44:55.577908 23731 external_mini_cluster.cc:1428] Setting key 591400cbfc6a8022b946c783562fd2de
I20260926 20:44:55.585906 11996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:55.579815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48477 (local address 127.0.0.1:39775)
0926 20:44:55.580705 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:55.580718 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:55.580761 (+ 43us) server_negotiation.cc:407] Connection header received
0926 20:44:55.582441 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:55.582465 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:55.582832 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:55.583257 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:55.584310 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:55.584316 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:55.584327 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:55.584340 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:55.584395 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:55.584419 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:55.584668 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:55.584804 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 20:44:55.584992 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":363,"threads_started":1}
I20260926 20:44:55.586342 11993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf: Bootstrap starting.
I20260926 20:44:55.589596 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:44:55.589706 11993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:55.590730 11993 log.cc:824] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:55.593649 11993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf: No bootstrap required, opened a new log
I20260926 20:44:55.597012 11996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:55.594084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34307 (local address 127.0.0.1:39775)
0926 20:44:55.594310 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:55.594316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:55.594666 (+ 350us) server_negotiation.cc:407] Connection header received
0926 20:44:55.594860 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:55.594864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:55.594921 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:55.595001 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:55.596195 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:55.596199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:55.596203 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:55.596220 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:55.596265 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:55.596284 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:55.596395 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:55.596628 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 20:44:55.596726 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 20:44:55.602538 11993 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a52b52da39407a92bdb0fc0266bfdf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39775 } }
I20260926 20:44:55.602981 11993 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:55.603094 11993 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2a52b52da39407a92bdb0fc0266bfdf, State: Initialized, Role: FOLLOWER
I20260926 20:44:55.603832 11993 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a52b52da39407a92bdb0fc0266bfdf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39775 } }
I20260926 20:44:55.604054 11993 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:44:55.604158 11993 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:44:55.604280 11993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:55.606498 11993 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a52b52da39407a92bdb0fc0266bfdf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39775 } }
I20260926 20:44:55.606956 11993 leader_election.cc:302] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2a52b52da39407a92bdb0fc0266bfdf; no voters:
I20260926 20:44:55.607587 11993 leader_election.cc:288] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:44:55.607754 12003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:55.608673 12003 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [term 1 LEADER]: Becoming Leader. State: Replica: d2a52b52da39407a92bdb0fc0266bfdf, State: Running, Role: LEADER
I20260926 20:44:55.609230 12003 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a52b52da39407a92bdb0fc0266bfdf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39775 } }
I20260926 20:44:55.609774 11993 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:44:55.612238 12005 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2a52b52da39407a92bdb0fc0266bfdf. Latest consensus state: current_term: 1 leader_uuid: "d2a52b52da39407a92bdb0fc0266bfdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a52b52da39407a92bdb0fc0266bfdf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39775 } } }
I20260926 20:44:55.612594 12004 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2a52b52da39407a92bdb0fc0266bfdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a52b52da39407a92bdb0fc0266bfdf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39775 } } }
I20260926 20:44:55.612860 12005 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:55.612985 12004 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:55.614724 12012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:44:55.619620 12012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:44:55.628588 12012 catalog_manager.cc:1347] Generated new cluster ID: 34552dd411d14760967510faed9f9f37
I20260926 20:44:55.628728 12012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:44:55.647198 12012 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:44:55.647693 12012 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:44:55.661321 12012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2a52b52da39407a92bdb0fc0266bfdf: Generated new TSK 0
I20260926 20:44:55.662343 12012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:44:55.674021 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 11926
W20260926 20:44:55.675278 11988 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48477
W20260926 20:44:55.676085 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 11926: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:44:55.676239 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 11926
2026-09-26T20:44:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2056 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:44:55 dist-test-slave-4d5t krb5kdc[12024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:44:55 dist-test-slave-4d5t krb5kdc[12024](info): set up 2 sockets
Sep 26 20:44:55 dist-test-slave-4d5t krb5kdc[12024](info): commencing operation
krb5kdc: starting...
W20260926 20:44:56.841351 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.004s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:44:57 dist-test-slave-4d5t krb5kdc[12024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:44:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:44:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:44:57.149449 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38621
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:57.507668 12037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:57.508044 12037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:57.508226 12037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:57.515934 12037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:44:57.516129 12037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:57.516250 12037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:57.516368 12037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:44:57.516458 12037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:44:57.526631 12037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:57.528028 12037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:57.529441 12037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:57.542712 12042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:57.542641 12043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:57.543298 12045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:57.543604 12037 server_base.cc:1034] running on GCE node
I20260926 20:44:57.544479 12037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:57.546047 12037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:57.547262 12037 hybrid_clock.cc:648] HybridClock initialized: now 1790455497547222 us; error 51 us; skew 500 ppm
I20260926 20:44:57.547607 12037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:57 dist-test-slave-4d5t krb5kdc[12024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455497, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:57.551007 12037 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:44:57.552783 12037 webserver.cc:460] Webserver started at http://127.0.0.1:46107/ using document root <none> and password file <none>
I20260926 20:44:57.553915 12037 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:57.554237 12037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:57.554654 12037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:57.557945 12037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "ce4ef06d6cd14c6f977a57ed834e4ee2"
format_stamp: "Formatted at 2026-09-26 20:44:57 on dist-test-slave-4d5t"
server_key: "b2fc92842e9cd2d72cc1948b9f93a82b"
server_key_iv: "be9b7edc4cbbcbbdb554c125ef7f7830"
server_key_version: "encryptionkey@0"
I20260926 20:44:57.558857 12037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "ce4ef06d6cd14c6f977a57ed834e4ee2"
format_stamp: "Formatted at 2026-09-26 20:44:57 on dist-test-slave-4d5t"
server_key: "b2fc92842e9cd2d72cc1948b9f93a82b"
server_key_iv: "be9b7edc4cbbcbbdb554c125ef7f7830"
server_key_version: "encryptionkey@0"
I20260926 20:44:57.564682 12037 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 20:44:57.569311 12052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:57.571055 12037 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260926 20:44:57.571220 12037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "ce4ef06d6cd14c6f977a57ed834e4ee2"
format_stamp: "Formatted at 2026-09-26 20:44:57 on dist-test-slave-4d5t"
server_key: "b2fc92842e9cd2d72cc1948b9f93a82b"
server_key_iv: "be9b7edc4cbbcbbdb554c125ef7f7830"
server_key_version: "encryptionkey@0"
I20260926 20:44:57.571411 12037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:57.607461 12037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:57.612438 12037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:57.612787 12037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:57.632999 12037 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38621
I20260926 20:44:57.633023 12103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38621 every 8 connection(s)
I20260926 20:44:57.634470 12037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:44:57.637979 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 12037
I20260926 20:44:57.638183 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:44:57.638715 23731 external_mini_cluster.cc:1428] Setting key 98d6b8ae04b6f8fd06ebbea1b5b98201
I20260926 20:44:57.638765 12104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:44:57 dist-test-slave-4d5t krb5kdc[12024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 20:44:57.656646 12104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2: Bootstrap starting.
I20260926 20:44:57.660610 12104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:57.661864 12104 log.cc:824] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:57.666435 12104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2: No bootstrap required, opened a new log
I20260926 20:44:57.667435 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:57.640893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42235 (local address 127.0.0.1:38621)
0926 20:44:57.641737 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:57.641750 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:57.641807 (+ 57us) server_negotiation.cc:407] Connection header received
0926 20:44:57.643634 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:57.643657 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:57.644082 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:57.644431 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:57.646850 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:57.648674 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:57.650930 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:57.652039 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:57.652240 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:57.657667 (+ 5427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:57.657676 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:57.657693 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:57.657743 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:57.661702 (+ 3959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:57.662824 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:57.662833 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:57.662841 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:57.663192 (+ 351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:57.663677 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:57.663681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:57.663683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:57.664243 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:57.664474 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:57.665394 (+ 920us) server_negotiation.cc:299] Negotiation successful
0926 20:44:57.665954 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":381,"threads_started":1}
I20260926 20:44:57.672821 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:44:57.678169 12104 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4ef06d6cd14c6f977a57ed834e4ee2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38621 } }
I20260926 20:44:57.678644 12104 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:57.678756 12104 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce4ef06d6cd14c6f977a57ed834e4ee2, State: Initialized, Role: FOLLOWER
I20260926 20:44:57.679421 12104 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4ef06d6cd14c6f977a57ed834e4ee2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38621 } }
I20260926 20:44:57.679692 12104 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:44:57.679845 12104 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:44:57.680023 12104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:57.682603 12104 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4ef06d6cd14c6f977a57ed834e4ee2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38621 } }
I20260926 20:44:57.683166 12104 leader_election.cc:302] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce4ef06d6cd14c6f977a57ed834e4ee2; no voters:
I20260926 20:44:57.683918 12104 leader_election.cc:288] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:44:57.684028 12114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:57.686496 12114 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [term 1 LEADER]: Becoming Leader. State: Replica: ce4ef06d6cd14c6f977a57ed834e4ee2, State: Running, Role: LEADER
I20260926 20:44:57.686681 12104 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:44:57.687230 12114 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4ef06d6cd14c6f977a57ed834e4ee2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38621 } }
I20260926 20:44:57.695299 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:57.677407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33573 (local address 127.0.0.1:38621)
0926 20:44:57.677607 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:57.677616 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:57.677984 (+ 368us) server_negotiation.cc:407] Connection header received
0926 20:44:57.678198 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:57.678204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:57.678285 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:57.678392 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:57.680351 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:57.681400 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:57.682739 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:57.683274 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:57.683416 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:57.686239 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:57.686249 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:57.686256 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:44:57.686311 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:57.690583 (+ 4272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:57.691499 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:57.691505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:57.691509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:57.691694 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:44:57.692159 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:44:57.692164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:44:57.692168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:44:57.692664 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:57.693373 (+ 709us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:57.694753 (+ 1380us) server_negotiation.cc:299] Negotiation successful
0926 20:44:57.694992 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 20:44:57.697537 12115 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce4ef06d6cd14c6f977a57ed834e4ee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4ef06d6cd14c6f977a57ed834e4ee2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38621 } } }
I20260926 20:44:57.698068 12115 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:57.698639 12116 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce4ef06d6cd14c6f977a57ed834e4ee2. Latest consensus state: current_term: 1 leader_uuid: "ce4ef06d6cd14c6f977a57ed834e4ee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce4ef06d6cd14c6f977a57ed834e4ee2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38621 } } }
I20260926 20:44:57.698956 12116 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2 [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:57.706219 12125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:44:57.711388 12125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:44:57.720726 12125 catalog_manager.cc:1347] Generated new cluster ID: 902c3826220243059d0cfbd47e35060b
I20260926 20:44:57.720868 12125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:44:57.732748 12125 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:44:57.734534 12125 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:44:57.743913 12125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce4ef06d6cd14c6f977a57ed834e4ee2: Generated new TSK 0
I20260926 20:44:57.745085 12125 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:44:57.761909 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12037
I20260926 20:44:57.763270 12099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:44:57.866676 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 12037
2026-09-26T20:44:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2188 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:44:57 dist-test-slave-4d5t krb5kdc[12137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:44:57 dist-test-slave-4d5t krb5kdc[12137](info): set up 2 sockets
Sep 26 20:44:57 dist-test-slave-4d5t krb5kdc[12137](info): commencing operation
krb5kdc: starting...
W20260926 20:44:59.017248 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:44:59 dist-test-slave-4d5t krb5kdc[12137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:44:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:44:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:44:59.305557 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34973
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:44:59.650935 12150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:44:59.651257 12150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:44:59.651383 12150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:44:59.659013 12150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:44:59.659147 12150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:44:59.659199 12150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:44:59.659305 12150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:44:59.659372 12150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:44:59.670121 12150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:44:59.671574 12150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:44:59.673059 12150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:44:59.686164 12155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:59.686244 12156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:44:59.686196 12158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:44:59.688349 12150 server_base.cc:1034] running on GCE node
I20260926 20:44:59.689308 12150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:44:59.691030 12150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:44:59.692310 12150 hybrid_clock.cc:648] HybridClock initialized: now 1790455499692249 us; error 58 us; skew 500 ppm
I20260926 20:44:59.692811 12150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:44:59 dist-test-slave-4d5t krb5kdc[12137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455499, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:44:59.696519 12150 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:44:59.698311 12150 webserver.cc:460] Webserver started at http://127.0.0.1:35469/ using document root <none> and password file <none>
I20260926 20:44:59.699061 12150 fs_manager.cc:359] Metadata directory not provided
I20260926 20:44:59.699172 12150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:44:59.699447 12150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:44:59.702929 12150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "ae1149a8503b4ff5a91adb8b2fd3e592"
format_stamp: "Formatted at 2026-09-26 20:44:59 on dist-test-slave-4d5t"
server_key: "7b01094999544d0690cca92b5770839c"
server_key_iv: "409489c34f8e8ef33b3435b626ef1126"
server_key_version: "encryptionkey@0"
I20260926 20:44:59.703759 12150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "ae1149a8503b4ff5a91adb8b2fd3e592"
format_stamp: "Formatted at 2026-09-26 20:44:59 on dist-test-slave-4d5t"
server_key: "7b01094999544d0690cca92b5770839c"
server_key_iv: "409489c34f8e8ef33b3435b626ef1126"
server_key_version: "encryptionkey@0"
I20260926 20:44:59.709605 12150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260926 20:44:59.713337 12165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:44:59.714839 12150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:44:59.715065 12150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "ae1149a8503b4ff5a91adb8b2fd3e592"
format_stamp: "Formatted at 2026-09-26 20:44:59 on dist-test-slave-4d5t"
server_key: "7b01094999544d0690cca92b5770839c"
server_key_iv: "409489c34f8e8ef33b3435b626ef1126"
server_key_version: "encryptionkey@0"
I20260926 20:44:59.715265 12150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:44:59.730489 12150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:44:59.735523 12150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:44:59.735883 12150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:44:59.758106 12150 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34973
I20260926 20:44:59.758129 12216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34973 every 8 connection(s)
I20260926 20:44:59.759588 12150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:44:59.763613 12217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:44:59.765552 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 12150
I20260926 20:44:59.765739 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:44:59.766268 23731 external_mini_cluster.cc:1428] Setting key 512b2363b37e672cbae683017d5aa9b6
I20260926 20:44:59.778672 12217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592: Bootstrap starting.
I20260926 20:44:59.780058 12220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:59.768272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34257 (local address 127.0.0.1:34973)
0926 20:44:59.769254 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:59.769267 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:59.769313 (+ 46us) server_negotiation.cc:407] Connection header received
0926 20:44:59.771040 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:59.771065 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:59.771518 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:59.771981 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:59.772597 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:59.773890 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:59.775287 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:59.776050 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:59.776285 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:59.777661 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:59.777670 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:59.777686 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:59.778251 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:59.778341 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:59.778367 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:59.778505 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:59.778734 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 20:44:59.779056 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":439,"threads_started":1}
I20260926 20:44:59.782909 12217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592: Neither blocks nor log segments found. Creating new log.
I20260926 20:44:59.784293 12217 log.cc:824] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592: Log is configured to *not* fsync() on all Append() calls
I20260926 20:44:59.785580 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:44:59.787762 12217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592: No bootstrap required, opened a new log
I20260926 20:44:59.796505 12217 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1149a8503b4ff5a91adb8b2fd3e592" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34973 } }
I20260926 20:44:59.797053 12217 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:44:59.797179 12217 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae1149a8503b4ff5a91adb8b2fd3e592, State: Initialized, Role: FOLLOWER
I20260926 20:44:59.797537 12220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:44:59.790656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39535 (local address 127.0.0.1:34973)
0926 20:44:59.790877 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 20:44:59.790891 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 20:44:59.791263 (+ 372us) server_negotiation.cc:407] Connection header received
0926 20:44:59.791534 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:44:59.791538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:44:59.791616 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:44:59.791745 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:44:59.792313 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:59.793237 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:59.794713 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:44:59.795249 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:44:59.795431 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:44:59.796685 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:44:59.796690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:44:59.796693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:44:59.796713 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:44:59.796755 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:44:59.796772 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:44:59.796887 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 20:44:59.797098 (+ 211us) server_negotiation.cc:299] Negotiation successful
0926 20:44:59.797215 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 20:44:59.797899 12217 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1149a8503b4ff5a91adb8b2fd3e592" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34973 } }
I20260926 20:44:59.798175 12217 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:44:59.798343 12217 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:44:59.798555 12217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:44:59.801035 12217 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1149a8503b4ff5a91adb8b2fd3e592" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34973 } }
I20260926 20:44:59.801671 12217 leader_election.cc:302] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae1149a8503b4ff5a91adb8b2fd3e592; no voters:
I20260926 20:44:59.802290 12217 leader_election.cc:288] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:44:59.802469 12227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:44:59.804188 12217 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:44:59.804620 12227 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [term 1 LEADER]: Becoming Leader. State: Replica: ae1149a8503b4ff5a91adb8b2fd3e592, State: Running, Role: LEADER
I20260926 20:44:59.805250 12227 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1149a8503b4ff5a91adb8b2fd3e592" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34973 } }
I20260926 20:44:59.808540 12229 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae1149a8503b4ff5a91adb8b2fd3e592. Latest consensus state: current_term: 1 leader_uuid: "ae1149a8503b4ff5a91adb8b2fd3e592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1149a8503b4ff5a91adb8b2fd3e592" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34973 } } }
I20260926 20:44:59.809000 12229 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:59.808789 12228 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae1149a8503b4ff5a91adb8b2fd3e592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1149a8503b4ff5a91adb8b2fd3e592" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34973 } } }
I20260926 20:44:59.809193 12228 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592 [sys.catalog]: This master's current role is: LEADER
I20260926 20:44:59.811239 12238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:44:59.817485 12238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:44:59.828269 12238 catalog_manager.cc:1347] Generated new cluster ID: ac049dd92ec741729da6513e79a334e5
I20260926 20:44:59.828423 12238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:44:59.850219 12238 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:44:59.852242 12238 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:44:59.866665 12238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae1149a8503b4ff5a91adb8b2fd3e592: Generated new TSK 0
I20260926 20:44:59.867849 12238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:44:59.883690 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12150
W20260926 20:44:59.884861 12212 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34257
W20260926 20:44:59.885744 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12150: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:44:59.885934 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 12150
2026-09-26T20:44:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2017 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:44:59 dist-test-slave-4d5t krb5kdc[12248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:44:59 dist-test-slave-4d5t krb5kdc[12248](info): set up 2 sockets
Sep 26 20:44:59 dist-test-slave-4d5t krb5kdc[12248](info): commencing operation
krb5kdc: starting...
W20260926 20:45:01.038358 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:01 dist-test-slave-4d5t krb5kdc[12248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:45:01.335803 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37989
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:01.679292 12261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:01.679625 12261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:01.679772 12261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:01.687668 12261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:01.687799 12261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:01.687894 12261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:01.687991 12261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:01.688084 12261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:01.698386 12261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:01.699783 12261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:01.701162 12261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:01.716980 12269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:01.716912 12266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:01.717027 12267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:01.717846 12261 server_base.cc:1034] running on GCE node
I20260926 20:45:01.718667 12261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:01.720319 12261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:01.721599 12261 hybrid_clock.cc:648] HybridClock initialized: now 1790455501721548 us; error 55 us; skew 500 ppm
I20260926 20:45:01.722043 12261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:01 dist-test-slave-4d5t krb5kdc[12248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455501, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:01.725598 12261 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:45:01.727337 12261 webserver.cc:460] Webserver started at http://127.0.0.1:40477/ using document root <none> and password file <none>
I20260926 20:45:01.728080 12261 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:01.728193 12261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:01.728588 12261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:01.731683 12261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "db3490ad12474418bf1fa60a21c96fee"
format_stamp: "Formatted at 2026-09-26 20:45:01 on dist-test-slave-4d5t"
server_key: "6371e0dfdc7a4991941f5c77e81bb70c"
server_key_iv: "ee5c27e50071f251172d15e7e5f33371"
server_key_version: "encryptionkey@0"
I20260926 20:45:01.732493 12261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "db3490ad12474418bf1fa60a21c96fee"
format_stamp: "Formatted at 2026-09-26 20:45:01 on dist-test-slave-4d5t"
server_key: "6371e0dfdc7a4991941f5c77e81bb70c"
server_key_iv: "ee5c27e50071f251172d15e7e5f33371"
server_key_version: "encryptionkey@0"
I20260926 20:45:01.738286 12261 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 20:45:01.742332 12276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:01.743999 12261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260926 20:45:01.744220 12261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "db3490ad12474418bf1fa60a21c96fee"
format_stamp: "Formatted at 2026-09-26 20:45:01 on dist-test-slave-4d5t"
server_key: "6371e0dfdc7a4991941f5c77e81bb70c"
server_key_iv: "ee5c27e50071f251172d15e7e5f33371"
server_key_version: "encryptionkey@0"
I20260926 20:45:01.744446 12261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:01.751176 12261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:01.757573 12261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:01.757925 12261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:01.778721 12261 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37989
I20260926 20:45:01.778775 12327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37989 every 8 connection(s)
I20260926 20:45:01.780272 12261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:01.783679 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 12261
I20260926 20:45:01.783900 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:01.784401 23731 external_mini_cluster.cc:1428] Setting key 495bcaf5f65063bbbe35765dc2319d26
I20260926 20:45:01.785058 12328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:01.795140 12331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:01.786777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40901 (local address 127.0.0.1:37989)
0926 20:45:01.787930 (+ 1153us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:01.787946 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:01.788002 (+ 56us) server_negotiation.cc:407] Connection header received
0926 20:45:01.790227 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:01.790263 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:01.790813 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:01.791354 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:01.792994 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:01.793003 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:01.793025 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:01.793046 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:01.793145 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:01.793175 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:01.793545 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:01.793720 (+ 175us) server_negotiation.cc:299] Negotiation successful
0926 20:45:01.793962 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831,"thread_start_us":494,"threads_started":1}
I20260926 20:45:01.799825 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:01.800957 12328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee: Bootstrap starting.
I20260926 20:45:01.805006 12328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:01.806265 12328 log.cc:824] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:01.808343 12331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:01.804471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55709 (local address 127.0.0.1:37989)
0926 20:45:01.805029 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:01.805037 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:01.805553 (+ 516us) server_negotiation.cc:407] Connection header received
0926 20:45:01.805980 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:01.805997 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:01.806088 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:01.806229 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:01.807607 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:01.807613 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:01.807616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:01.807640 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:01.807688 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:01.807706 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:01.807867 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:01.807994 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 20:45:01.808065 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260926 20:45:01.809799 12328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee: No bootstrap required, opened a new log
I20260926 20:45:01.817567 12328 raft_consensus.cc:348] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3490ad12474418bf1fa60a21c96fee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37989 } }
I20260926 20:45:01.818066 12328 raft_consensus.cc:374] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:01.818188 12328 raft_consensus.cc:729] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3490ad12474418bf1fa60a21c96fee, State: Initialized, Role: FOLLOWER
I20260926 20:45:01.818809 12328 consensus_queue.cc:260] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3490ad12474418bf1fa60a21c96fee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37989 } }
I20260926 20:45:01.819083 12328 raft_consensus.cc:388] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:01.819242 12328 raft_consensus.cc:482] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:01.819429 12328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:01.822886 12328 raft_consensus.cc:504] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3490ad12474418bf1fa60a21c96fee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37989 } }
I20260926 20:45:01.823575 12328 leader_election.cc:302] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db3490ad12474418bf1fa60a21c96fee; no voters:
I20260926 20:45:01.824322 12328 leader_election.cc:288] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:01.824728 12338 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:01.825711 12338 raft_consensus.cc:686] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [term 1 LEADER]: Becoming Leader. State: Replica: db3490ad12474418bf1fa60a21c96fee, State: Running, Role: LEADER
I20260926 20:45:01.826478 12338 consensus_queue.cc:237] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3490ad12474418bf1fa60a21c96fee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37989 } }
I20260926 20:45:01.826834 12328 sys_catalog.cc:564] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:01.830739 12339 sys_catalog.cc:455] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db3490ad12474418bf1fa60a21c96fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3490ad12474418bf1fa60a21c96fee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37989 } } }
I20260926 20:45:01.830791 12340 sys_catalog.cc:455] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [sys.catalog]: SysCatalogTable state changed. Reason: New leader db3490ad12474418bf1fa60a21c96fee. Latest consensus state: current_term: 1 leader_uuid: "db3490ad12474418bf1fa60a21c96fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3490ad12474418bf1fa60a21c96fee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37989 } } }
I20260926 20:45:01.831483 12339 sys_catalog.cc:458] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:01.831492 12340 sys_catalog.cc:458] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:01.832929 12347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:01.840781 12347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:01.851679 12347 catalog_manager.cc:1347] Generated new cluster ID: 5f4dded517de4bba8ad02bdc1061c356
I20260926 20:45:01.851903 12347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:01.870054 12347 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:01.870639 12347 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:01.891129 12347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db3490ad12474418bf1fa60a21c96fee: Generated new TSK 0
I20260926 20:45:01.892135 12347 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:01.903083 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12261
W20260926 20:45:01.904466 12323 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40901
W20260926 20:45:01.905354 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12261: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:45:01.905503 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 12261
2026-09-26T20:45:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2019 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:02 dist-test-slave-4d5t krb5kdc[12359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:02 dist-test-slave-4d5t krb5kdc[12359](info): set up 2 sockets
Sep 26 20:45:02 dist-test-slave-4d5t krb5kdc[12359](info): commencing operation
krb5kdc: starting...
W20260926 20:45:03.059275 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:03 dist-test-slave-4d5t krb5kdc[12359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:45:03.345853 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44987
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:03.686908 12372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:03.687217 12372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:03.687350 12372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:03.695591 12372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:03.695739 12372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:03.695850 12372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:03.695973 12372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:03.696054 12372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:03.706583 12372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:03.707983 12372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:03.709368 12372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:03.721822 12380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:03.721846 12378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:03.721884 12377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:03.722682 12372 server_base.cc:1034] running on GCE node
I20260926 20:45:03.723459 12372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:03.724913 12372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:03.726125 12372 hybrid_clock.cc:648] HybridClock initialized: now 1790455503726090 us; error 30 us; skew 500 ppm
I20260926 20:45:03.726516 12372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:03 dist-test-slave-4d5t krb5kdc[12359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455503, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:03.733951 12372 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:45:03.735615 12372 webserver.cc:460] Webserver started at http://127.0.0.1:36759/ using document root <none> and password file <none>
I20260926 20:45:03.736327 12372 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:03.736486 12372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:03.736824 12372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:03.739979 12372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "f8ac76131b1e48a3a658d15f7a26050b"
format_stamp: "Formatted at 2026-09-26 20:45:03 on dist-test-slave-4d5t"
server_key: "9d545587719f2162f6bc5495a29ff834"
server_key_iv: "0e2a8ca4b2585c4f269e6f10bfa1a6e6"
server_key_version: "encryptionkey@0"
I20260926 20:45:03.740927 12372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "f8ac76131b1e48a3a658d15f7a26050b"
format_stamp: "Formatted at 2026-09-26 20:45:03 on dist-test-slave-4d5t"
server_key: "9d545587719f2162f6bc5495a29ff834"
server_key_iv: "0e2a8ca4b2585c4f269e6f10bfa1a6e6"
server_key_version: "encryptionkey@0"
I20260926 20:45:03.748268 12372 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260926 20:45:03.752158 12387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:03.753711 12372 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:45:03.753944 12372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "f8ac76131b1e48a3a658d15f7a26050b"
format_stamp: "Formatted at 2026-09-26 20:45:03 on dist-test-slave-4d5t"
server_key: "9d545587719f2162f6bc5495a29ff834"
server_key_iv: "0e2a8ca4b2585c4f269e6f10bfa1a6e6"
server_key_version: "encryptionkey@0"
I20260926 20:45:03.754182 12372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:03.778915 12372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:03.783679 12372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:03.784001 12372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:03.805804 12372 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44987
I20260926 20:45:03.805838 12438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44987 every 8 connection(s)
I20260926 20:45:03.807240 12372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:03.811069 12439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:03.813512 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 12372
I20260926 20:45:03.813740 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:03.814239 23731 external_mini_cluster.cc:1428] Setting key b77e7fad5bb50b48dc967ebf88b5d21e
I20260926 20:45:03.824437 12439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b: Bootstrap starting.
I20260926 20:45:03.828297 12439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:03.829452 12439 log.cc:824] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b: Log is configured to *not* fsync() on all Append() calls
Sep 26 20:45:03 dist-test-slave-4d5t krb5kdc[12359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 20:45:03.832993 12439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b: No bootstrap required, opened a new log
I20260926 20:45:03.840214 12442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:03.816173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:51303 (local address 127.0.0.1:44987)
0926 20:45:03.817024 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:03.817046 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:03.817096 (+ 50us) server_negotiation.cc:407] Connection header received
0926 20:45:03.819464 (+ 2368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:03.819496 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:03.820037 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:03.820517 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:03.822003 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:03.823939 (+ 1936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:03.825374 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:03.826524 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:03.826724 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:03.831587 (+ 4863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:03.831604 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:03.831621 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:03.831669 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:03.835070 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:03.836203 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:03.836211 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:03.836217 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:03.836418 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:03.837240 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:03.837244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:03.837247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:03.837826 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:03.838123 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:03.838632 (+ 509us) server_negotiation.cc:299] Negotiation successful
0926 20:45:03.839096 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":359,"threads_started":1}
I20260926 20:45:03.842329 12439 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ac76131b1e48a3a658d15f7a26050b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44987 } }
I20260926 20:45:03.842762 12439 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:03.842883 12439 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8ac76131b1e48a3a658d15f7a26050b, State: Initialized, Role: FOLLOWER
I20260926 20:45:03.843559 12439 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ac76131b1e48a3a658d15f7a26050b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44987 } }
I20260926 20:45:03.843839 12439 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:03.843997 12439 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:03.844172 12439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:03.844476 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:03.846738 12439 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ac76131b1e48a3a658d15f7a26050b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44987 } }
I20260926 20:45:03.847306 12439 leader_election.cc:302] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8ac76131b1e48a3a658d15f7a26050b; no voters:
I20260926 20:45:03.848026 12439 leader_election.cc:288] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:03.848124 12447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:03.850080 12447 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [term 1 LEADER]: Becoming Leader. State: Replica: f8ac76131b1e48a3a658d15f7a26050b, State: Running, Role: LEADER
I20260926 20:45:03.850636 12439 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:03.850693 12447 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ac76131b1e48a3a658d15f7a26050b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44987 } }
I20260926 20:45:03.855193 12450 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8ac76131b1e48a3a658d15f7a26050b. Latest consensus state: current_term: 1 leader_uuid: "f8ac76131b1e48a3a658d15f7a26050b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ac76131b1e48a3a658d15f7a26050b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44987 } } }
I20260926 20:45:03.855685 12450 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:03.856223 12449 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8ac76131b1e48a3a658d15f7a26050b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ac76131b1e48a3a658d15f7a26050b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44987 } } }
I20260926 20:45:03.856506 12449 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:03.861963 12456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:03.866804 12442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:03.849726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:39835 (local address 127.0.0.1:44987)
0926 20:45:03.849947 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:03.849956 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:03.849985 (+ 29us) server_negotiation.cc:407] Connection header received
0926 20:45:03.850176 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:03.850185 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:03.850315 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:03.850392 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:03.852251 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:03.853678 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:03.855730 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:03.856507 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:03.856642 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:03.859840 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:03.859845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:03.859849 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:03.859885 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:03.863300 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:03.864163 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:03.864169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:03.864172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:03.864335 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:03.864983 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:03.864989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:03.864992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:03.865475 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:03.865665 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:03.866230 (+ 565us) server_negotiation.cc:299] Negotiation successful
0926 20:45:03.866421 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:45:03.876039 12456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:03.886020 12456 catalog_manager.cc:1347] Generated new cluster ID: cde152054f0f41d3a3bded8ec63147f8
I20260926 20:45:03.886193 12456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:03.897783 12456 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:03.899191 12456 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:03.905808 12456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8ac76131b1e48a3a658d15f7a26050b: Generated new TSK 0
I20260926 20:45:03.906736 12456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:03.914727 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12372
I20260926 20:45:03.916133 12434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:04.015372 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 12372
W20260926 20:45:04.033732 12357 connection.cc:535] client connection to 127.0.0.1:44987 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44987): Connection reset by peer (error 104)
2026-09-26T20:45:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2111 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:04 dist-test-slave-4d5t krb5kdc[12471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:04 dist-test-slave-4d5t krb5kdc[12471](info): set up 2 sockets
Sep 26 20:45:04 dist-test-slave-4d5t krb5kdc[12471](info): commencing operation
krb5kdc: starting...
W20260926 20:45:05.189296 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.001s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:05 dist-test-slave-4d5t krb5kdc[12471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:45:05.479758 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39737
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:05.839494 12484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:05.839834 12484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:05.839991 12484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:05.848990 12484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:05.849153 12484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:05.849205 12484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:05.849313 12484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:05.849383 12484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:05.861443 12484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:05.862960 12484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:05.864339 12484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:05.877205 12492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:05.877303 12490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:05.877238 12489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:05.878414 12484 server_base.cc:1034] running on GCE node
I20260926 20:45:05.879145 12484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:05.880657 12484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:05.881876 12484 hybrid_clock.cc:648] HybridClock initialized: now 1790455505881818 us; error 44 us; skew 500 ppm
I20260926 20:45:05.882273 12484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:05 dist-test-slave-4d5t krb5kdc[12471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455505, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:05.885468 12484 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:45:05.886935 12484 webserver.cc:460] Webserver started at http://127.0.0.1:45227/ using document root <none> and password file <none>
I20260926 20:45:05.887562 12484 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:05.887661 12484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:05.887897 12484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:05.890977 12484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "c05074a56e06401b9f326a8560e47ef4"
format_stamp: "Formatted at 2026-09-26 20:45:05 on dist-test-slave-4d5t"
server_key: "1d3c2415820cab59e2e84a9bf2c395de"
server_key_iv: "0ed54b9186ccadb63edc13462a518303"
server_key_version: "encryptionkey@0"
I20260926 20:45:05.891737 12484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "c05074a56e06401b9f326a8560e47ef4"
format_stamp: "Formatted at 2026-09-26 20:45:05 on dist-test-slave-4d5t"
server_key: "1d3c2415820cab59e2e84a9bf2c395de"
server_key_iv: "0ed54b9186ccadb63edc13462a518303"
server_key_version: "encryptionkey@0"
I20260926 20:45:05.897061 12484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 20:45:05.901176 12499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:05.902889 12484 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 20:45:05.903124 12484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "c05074a56e06401b9f326a8560e47ef4"
format_stamp: "Formatted at 2026-09-26 20:45:05 on dist-test-slave-4d5t"
server_key: "1d3c2415820cab59e2e84a9bf2c395de"
server_key_iv: "0ed54b9186ccadb63edc13462a518303"
server_key_version: "encryptionkey@0"
I20260926 20:45:05.903363 12484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:05.922240 12484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:05.927003 12484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:05.927342 12484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:05.949679 12484 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39737
I20260926 20:45:05.949702 12550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39737 every 8 connection(s)
I20260926 20:45:05.951242 12484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:05.955940 12551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:05.959220 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 12484
I20260926 20:45:05.959403 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:05.959932 23731 external_mini_cluster.cc:1428] Setting key 37160e3fa8268173c8c260b1d8e9bff4
I20260926 20:45:05.968894 12551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4: Bootstrap starting.
I20260926 20:45:05.972208 12551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:05.973538 12551 log.cc:824] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:05.974056 12554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:05.962102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:52347 (local address 127.0.0.1:39737)
0926 20:45:05.962992 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:05.963005 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:05.963058 (+ 53us) server_negotiation.cc:407] Connection header received
0926 20:45:05.964874 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:05.964898 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:05.965332 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:05.965760 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:05.966337 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:05.967672 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:05.969275 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:05.969945 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:05.970159 (+ 214us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:05.971579 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:05.971588 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:05.971601 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:05.972154 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:05.972237 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:05.972269 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:05.972496 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:05.972828 (+ 332us) server_negotiation.cc:299] Negotiation successful
0926 20:45:05.972997 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":415,"threads_started":1}
I20260926 20:45:05.976538 12551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4: No bootstrap required, opened a new log
I20260926 20:45:05.977890 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:05.985569 12551 raft_consensus.cc:348] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05074a56e06401b9f326a8560e47ef4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39737 } }
I20260926 20:45:05.986011 12551 raft_consensus.cc:374] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:05.986132 12551 raft_consensus.cc:729] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c05074a56e06401b9f326a8560e47ef4, State: Initialized, Role: FOLLOWER
I20260926 20:45:05.986867 12551 consensus_queue.cc:260] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05074a56e06401b9f326a8560e47ef4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39737 } }
I20260926 20:45:05.987183 12551 raft_consensus.cc:388] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:05.987334 12551 raft_consensus.cc:482] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:05.987550 12551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:05.989804 12554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:05.982383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:53217 (local address 127.0.0.1:39737)
0926 20:45:05.982573 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:05.982582 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:05.982902 (+ 320us) server_negotiation.cc:407] Connection header received
0926 20:45:05.983059 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:05.983066 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:05.983163 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:05.983284 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:05.983766 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:05.984956 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:05.986710 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:05.987420 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:05.987579 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:05.988937 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:05.988943 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:05.988948 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:05.988972 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:05.989036 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:05.989063 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:05.989183 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:05.989416 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 20:45:05.989474 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 20:45:05.990799 12551 raft_consensus.cc:504] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05074a56e06401b9f326a8560e47ef4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39737 } }
I20260926 20:45:05.991338 12551 leader_election.cc:302] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c05074a56e06401b9f326a8560e47ef4; no voters:
I20260926 20:45:05.992065 12551 leader_election.cc:288] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:05.992249 12561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:05.993079 12561 raft_consensus.cc:686] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [term 1 LEADER]: Becoming Leader. State: Replica: c05074a56e06401b9f326a8560e47ef4, State: Running, Role: LEADER
I20260926 20:45:05.994042 12561 consensus_queue.cc:237] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05074a56e06401b9f326a8560e47ef4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39737 } }
I20260926 20:45:05.994344 12551 sys_catalog.cc:564] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:05.997499 12563 sys_catalog.cc:455] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c05074a56e06401b9f326a8560e47ef4. Latest consensus state: current_term: 1 leader_uuid: "c05074a56e06401b9f326a8560e47ef4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05074a56e06401b9f326a8560e47ef4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39737 } } }
I20260926 20:45:05.997933 12563 sys_catalog.cc:458] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:05.998839 12568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:05.999260 12562 sys_catalog.cc:455] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c05074a56e06401b9f326a8560e47ef4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05074a56e06401b9f326a8560e47ef4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39737 } } }
I20260926 20:45:05.999573 12562 sys_catalog.cc:458] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:06.004434 12568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:06.016024 12568 catalog_manager.cc:1347] Generated new cluster ID: b6ce06bb42484235b2e365f6c67b20ea
I20260926 20:45:06.016165 12568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:06.036403 12568 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:06.038347 12568 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:06.045125 12568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c05074a56e06401b9f326a8560e47ef4: Generated new TSK 0
I20260926 20:45:06.046533 12568 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:06.054152 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12484
W20260926 20:45:06.056157 12546 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.115:52347
W20260926 20:45:06.057066 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12484: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:45:06.057238 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 12484
W20260926 20:45:06.074863 12469 connection.cc:535] client connection to 127.0.0.1:39737 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39737): Connection reset by peer (error 104)
2026-09-26T20:45:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2040 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:06 dist-test-slave-4d5t krb5kdc[12582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:06 dist-test-slave-4d5t krb5kdc[12582](info): set up 2 sockets
Sep 26 20:45:06 dist-test-slave-4d5t krb5kdc[12582](info): commencing operation
krb5kdc: starting...
W20260926 20:45:07.229277 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:07 dist-test-slave-4d5t krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:45:07.517277 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45801
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:07.882406 12595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:07.882730 12595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:07.882906 12595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:07.890378 12595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:07.890508 12595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:07.890596 12595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:07.890684 12595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:07.890725 12595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:07.900959 12595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:07.902374 12595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:07.903702 12595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:07.916479 12601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:07.916517 12603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:07.916599 12600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:07.917438 12595 server_base.cc:1034] running on GCE node
I20260926 20:45:07.918357 12595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:07.919951 12595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:07.921206 12595 hybrid_clock.cc:648] HybridClock initialized: now 1790455507921156 us; error 48 us; skew 500 ppm
I20260926 20:45:07.921622 12595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:07 dist-test-slave-4d5t krb5kdc[12582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455507, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:07.925017 12595 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:45:07.926604 12595 webserver.cc:460] Webserver started at http://127.0.0.1:39753/ using document root <none> and password file <none>
I20260926 20:45:07.927486 12595 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:07.927596 12595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:07.927843 12595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:07.930871 12595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "c5bdbe1bb01f4207984a65a5a15162e0"
format_stamp: "Formatted at 2026-09-26 20:45:07 on dist-test-slave-4d5t"
server_key: "bb0cf07833dbfc7331e846a8e35972f0"
server_key_iv: "63ae225a534055d94d61ba4c05d385a4"
server_key_version: "encryptionkey@0"
I20260926 20:45:07.931617 12595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "c5bdbe1bb01f4207984a65a5a15162e0"
format_stamp: "Formatted at 2026-09-26 20:45:07 on dist-test-slave-4d5t"
server_key: "bb0cf07833dbfc7331e846a8e35972f0"
server_key_iv: "63ae225a534055d94d61ba4c05d385a4"
server_key_version: "encryptionkey@0"
I20260926 20:45:07.937000 12595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 20:45:07.940802 12610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:07.942394 12595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:45:07.942601 12595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "c5bdbe1bb01f4207984a65a5a15162e0"
format_stamp: "Formatted at 2026-09-26 20:45:07 on dist-test-slave-4d5t"
server_key: "bb0cf07833dbfc7331e846a8e35972f0"
server_key_iv: "63ae225a534055d94d61ba4c05d385a4"
server_key_version: "encryptionkey@0"
I20260926 20:45:07.942808 12595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:07.950106 12595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:07.955042 12595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:07.955374 12595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:07.975804 12595 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45801
I20260926 20:45:07.975845 12661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45801 every 8 connection(s)
I20260926 20:45:07.977334 12595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:07.978266 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 12595
I20260926 20:45:07.978468 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:07.978986 23731 external_mini_cluster.cc:1428] Setting key 9126da5219f1d6591bc26c82c97358da
I20260926 20:45:07.983861 12662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:07.988813 12665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:07.981101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:51889 (local address 127.0.0.1:45801)
0926 20:45:07.982262 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:07.982304 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:07.982377 (+ 73us) server_negotiation.cc:407] Connection header received
0926 20:45:07.984880 (+ 2503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:07.984908 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:07.985355 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:07.985822 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:07.986791 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:07.986799 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:07.986814 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:07.986833 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:07.986932 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:07.986965 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:07.987200 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:07.987339 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 20:45:07.987558 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":448,"threads_started":1}
I20260926 20:45:07.993826 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:07.997308 12662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0: Bootstrap starting.
I20260926 20:45:08.000990 12662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:08.001055 12665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:07.998414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:41487 (local address 127.0.0.1:45801)
0926 20:45:07.998625 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:07.998633 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:07.998993 (+ 360us) server_negotiation.cc:407] Connection header received
0926 20:45:07.999208 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:07.999214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:07.999277 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:07.999390 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:08.000346 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:08.000352 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:08.000357 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:08.000376 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:08.000425 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:08.000442 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:08.000563 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:08.000696 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 20:45:08.000763 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:45:08.002177 12662 log.cc:824] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:08.005016 12662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0: No bootstrap required, opened a new log
I20260926 20:45:08.013233 12662 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bdbe1bb01f4207984a65a5a15162e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45801 } }
I20260926 20:45:08.013733 12662 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:08.013898 12662 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bdbe1bb01f4207984a65a5a15162e0, State: Initialized, Role: FOLLOWER
I20260926 20:45:08.014626 12662 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bdbe1bb01f4207984a65a5a15162e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45801 } }
I20260926 20:45:08.014869 12662 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:08.014988 12662 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:08.015134 12662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:08.017603 12662 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bdbe1bb01f4207984a65a5a15162e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45801 } }
I20260926 20:45:08.018126 12662 leader_election.cc:302] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5bdbe1bb01f4207984a65a5a15162e0; no voters:
I20260926 20:45:08.018781 12662 leader_election.cc:288] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:08.019001 12672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:08.019701 12672 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [term 1 LEADER]: Becoming Leader. State: Replica: c5bdbe1bb01f4207984a65a5a15162e0, State: Running, Role: LEADER
I20260926 20:45:08.020370 12672 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bdbe1bb01f4207984a65a5a15162e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45801 } }
I20260926 20:45:08.021142 12662 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:08.023088 12673 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5bdbe1bb01f4207984a65a5a15162e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bdbe1bb01f4207984a65a5a15162e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45801 } } }
I20260926 20:45:08.023484 12673 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:08.024416 12677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:08.024259 12674 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5bdbe1bb01f4207984a65a5a15162e0. Latest consensus state: current_term: 1 leader_uuid: "c5bdbe1bb01f4207984a65a5a15162e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5bdbe1bb01f4207984a65a5a15162e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45801 } } }
I20260926 20:45:08.024605 12674 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:08.031208 12677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:08.041457 12677 catalog_manager.cc:1347] Generated new cluster ID: cf8eb22609d5428e8edfb7b7499886a3
I20260926 20:45:08.041617 12677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:08.060702 12677 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:08.061275 12677 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:08.084714 12677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5bdbe1bb01f4207984a65a5a15162e0: Generated new TSK 0
I20260926 20:45:08.085912 12677 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:08.090132 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12595
W20260926 20:45:08.091408 12657 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.115:51889
W20260926 20:45:08.092260 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12595: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:45:08.092542 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 12595
2026-09-26T20:45:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2036 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:08 dist-test-slave-4d5t krb5kdc[12693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:08 dist-test-slave-4d5t krb5kdc[12693](info): set up 2 sockets
Sep 26 20:45:08 dist-test-slave-4d5t krb5kdc[12693](info): commencing operation
krb5kdc: starting...
W20260926 20:45:09.225414 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:09 dist-test-slave-4d5t krb5kdc[12693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:45:09.523756 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37453
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:09.881665 12706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:09.882011 12706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:09.882117 12706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:09.890918 12706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:09.891067 12706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:09.891119 12706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:09.891180 12706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:09.891227 12706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:09.902172 12706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37453
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:09.903501 12706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:09.905089 12706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:09.917914 12711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:09.917835 12712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:09.918385 12714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:09.918828 12706 server_base.cc:1034] running on GCE node
I20260926 20:45:09.919662 12706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:09.921521 12706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:09.922765 12706 hybrid_clock.cc:648] HybridClock initialized: now 1790455509922734 us; error 44 us; skew 500 ppm
I20260926 20:45:09.923164 12706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:09 dist-test-slave-4d5t krb5kdc[12693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455509, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:09.936127 12706 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:45:09.938046 12706 webserver.cc:460] Webserver started at http://127.0.0.1:40003/ using document root <none> and password file <none>
I20260926 20:45:09.938799 12706 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:09.938941 12706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:09.939278 12706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:09.942559 12706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "380afac62dc04fcc826bd9a08540d577"
format_stamp: "Formatted at 2026-09-26 20:45:09 on dist-test-slave-4d5t"
server_key: "1f78b7f188178ff4bc05c970e7909786"
server_key_iv: "8e17168c17e39679672fa10c6e046b91"
server_key_version: "encryptionkey@0"
I20260926 20:45:09.943401 12706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "380afac62dc04fcc826bd9a08540d577"
format_stamp: "Formatted at 2026-09-26 20:45:09 on dist-test-slave-4d5t"
server_key: "1f78b7f188178ff4bc05c970e7909786"
server_key_iv: "8e17168c17e39679672fa10c6e046b91"
server_key_version: "encryptionkey@0"
I20260926 20:45:09.949728 12706 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260926 20:45:09.954067 12721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:09.955920 12706 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260926 20:45:09.956180 12706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "380afac62dc04fcc826bd9a08540d577"
format_stamp: "Formatted at 2026-09-26 20:45:09 on dist-test-slave-4d5t"
server_key: "1f78b7f188178ff4bc05c970e7909786"
server_key_iv: "8e17168c17e39679672fa10c6e046b91"
server_key_version: "encryptionkey@0"
I20260926 20:45:09.956436 12706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:09.975792 12706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:09.981508 12706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:09.981873 12706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:10.004917 12706 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37453
I20260926 20:45:10.004962 12772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37453 every 8 connection(s)
I20260926 20:45:10.006392 12706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:10.010329 12773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:10.013737 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 12706
I20260926 20:45:10.013921 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:10.014441 23731 external_mini_cluster.cc:1428] Setting key 35529ddba23da5de962fe35acdbabdac
I20260926 20:45:10.023698 12773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577: Bootstrap starting.
I20260926 20:45:10.028285 12773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:10.029780 12773 log.cc:824] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577: Log is configured to *not* fsync() on all Append() calls
Sep 26 20:45:10 dist-test-slave-4d5t krb5kdc[12693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 20:45:10.034154 12773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577: No bootstrap required, opened a new log
I20260926 20:45:10.041623 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:10.016533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:45281 (local address 127.0.0.1:37453)
0926 20:45:10.017367 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:10.017379 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:10.017422 (+ 43us) server_negotiation.cc:407] Connection header received
0926 20:45:10.019738 (+ 2316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:10.019782 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:10.020889 (+ 1107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:10.021405 (+ 516us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:10.022985 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:10.024453 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:10.025897 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:10.026643 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:10.026865 (+ 222us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:10.032539 (+ 5674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:10.032547 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:10.032637 (+ 90us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:10.032679 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:10.036357 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:10.037470 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:10.037481 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:10.037488 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:10.037673 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:10.038187 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:10.038191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:10.038193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:10.038742 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:10.038998 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:10.039530 (+ 532us) server_negotiation.cc:299] Negotiation successful
0926 20:45:10.039915 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":348,"threads_started":1}
I20260926 20:45:10.043677 12773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380afac62dc04fcc826bd9a08540d577" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37453 } }
I20260926 20:45:10.044101 12773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:10.044232 12773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380afac62dc04fcc826bd9a08540d577, State: Initialized, Role: FOLLOWER
I20260926 20:45:10.044886 12773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380afac62dc04fcc826bd9a08540d577" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37453 } }
I20260926 20:45:10.045186 12773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:10.045332 12773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:10.045527 12773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:10.046590 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:10.048725 12773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380afac62dc04fcc826bd9a08540d577" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37453 } }
I20260926 20:45:10.049376 12773 leader_election.cc:302] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 380afac62dc04fcc826bd9a08540d577; no voters:
I20260926 20:45:10.050191 12773 leader_election.cc:288] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:10.050400 12778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:10.051218 12778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [term 1 LEADER]: Becoming Leader. State: Replica: 380afac62dc04fcc826bd9a08540d577, State: Running, Role: LEADER
I20260926 20:45:10.051975 12778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380afac62dc04fcc826bd9a08540d577" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37453 } }
I20260926 20:45:10.054078 12778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 380afac62dc04fcc826bd9a08540d577. Latest consensus state: current_term: 1 leader_uuid: "380afac62dc04fcc826bd9a08540d577" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380afac62dc04fcc826bd9a08540d577" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37453 } } }
I20260926 20:45:10.054476 12778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:10.055012 12782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "380afac62dc04fcc826bd9a08540d577" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380afac62dc04fcc826bd9a08540d577" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37453 } } }
I20260926 20:45:10.055485 12782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:10.064913 12773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:10.065167 12784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:10.071233 12784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:10.087679 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:10.065139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:43155 (local address 127.0.0.1:37453)
0926 20:45:10.065336 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:10.065342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:10.066026 (+ 684us) server_negotiation.cc:407] Connection header received
0926 20:45:10.066224 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:10.066230 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:10.066336 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:10.066478 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:10.068888 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:10.070126 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:10.071920 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:10.072710 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:10.072822 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:10.077541 (+ 4719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:10.077551 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:10.077557 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:10.077621 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:10.081892 (+ 4271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:10.083446 (+ 1554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:10.083456 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:10.083461 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:10.083673 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:10.084469 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:10.084478 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:10.084483 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:10.085039 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:10.085390 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:10.086846 (+ 1456us) server_negotiation.cc:299] Negotiation successful
0926 20:45:10.087126 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 20:45:10.091398 12784 catalog_manager.cc:1347] Generated new cluster ID: f3fa30aca1bf476f9ca9e0beba60d864
I20260926 20:45:10.091574 12784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:10.104591 12784 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:10.106654 12784 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:10.121255 12784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 380afac62dc04fcc826bd9a08540d577: Generated new TSK 0
I20260926 20:45:10.122393 12784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:10.135600 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12706
I20260926 20:45:10.137238 12768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:10.228869 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 12706
W20260926 20:45:10.247854 12691 connection.cc:535] client connection to 127.0.0.1:37453 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37453): Connection reset by peer (error 104)
2026-09-26T20:45:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2136 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:10 dist-test-slave-4d5t krb5kdc[12805](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:10 dist-test-slave-4d5t krb5kdc[12805](info): set up 2 sockets
Sep 26 20:45:10 dist-test-slave-4d5t krb5kdc[12805](info): commencing operation
krb5kdc: starting...
W20260926 20:45:11.397229 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:11 dist-test-slave-4d5t krb5kdc[12805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:45:11.692018 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34719
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:12.051517 12818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:12.051963 12818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:12.052151 12818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:12.061092 12818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:12.061276 12818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:12.061381 12818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:12.061503 12818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:12.061581 12818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:12.073525 12818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34719
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:12.075069 12818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:12.076443 12818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:12.090513 12824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:12.090603 12823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:12.090603 12826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:12.092703 12818 server_base.cc:1034] running on GCE node
I20260926 20:45:12.093628 12818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:12.095283 12818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:12.096531 12818 hybrid_clock.cc:648] HybridClock initialized: now 1790455512096467 us; error 57 us; skew 500 ppm
I20260926 20:45:12.096972 12818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:12 dist-test-slave-4d5t krb5kdc[12805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455512, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:12.100373 12818 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:45:12.102092 12818 webserver.cc:460] Webserver started at http://127.0.0.1:33299/ using document root <none> and password file <none>
I20260926 20:45:12.102840 12818 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:12.102993 12818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:12.103305 12818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:12.106398 12818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "d24c0526d4ee4ccdbb34515f354e0551"
format_stamp: "Formatted at 2026-09-26 20:45:12 on dist-test-slave-4d5t"
server_key: "1b921596c0e200d13f16b486fb548288"
server_key_iv: "622c835c1bc40aeed18e9152b8bd7dcc"
server_key_version: "encryptionkey@0"
I20260926 20:45:12.107259 12818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "d24c0526d4ee4ccdbb34515f354e0551"
format_stamp: "Formatted at 2026-09-26 20:45:12 on dist-test-slave-4d5t"
server_key: "1b921596c0e200d13f16b486fb548288"
server_key_iv: "622c835c1bc40aeed18e9152b8bd7dcc"
server_key_version: "encryptionkey@0"
I20260926 20:45:12.112912 12818 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 20:45:12.116600 12833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:12.118068 12818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:45:12.118281 12818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "d24c0526d4ee4ccdbb34515f354e0551"
format_stamp: "Formatted at 2026-09-26 20:45:12 on dist-test-slave-4d5t"
server_key: "1b921596c0e200d13f16b486fb548288"
server_key_iv: "622c835c1bc40aeed18e9152b8bd7dcc"
server_key_version: "encryptionkey@0"
I20260926 20:45:12.118502 12818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:12.148545 12818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:12.153365 12818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:12.153695 12818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:12.176755 12818 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34719
I20260926 20:45:12.176759 12884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34719 every 8 connection(s)
I20260926 20:45:12.178280 12818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:12.181734 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 12818
I20260926 20:45:12.181911 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:12.182395 12885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:12.182489 23731 external_mini_cluster.cc:1428] Setting key 31b83fbceac82afb153c9eacd17ea8a2
I20260926 20:45:12.195834 12885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551: Bootstrap starting.
I20260926 20:45:12.197093 12888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:12.184372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:58245 (local address 127.0.0.1:34719)
0926 20:45:12.185660 (+ 1288us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:12.185673 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:12.185722 (+ 49us) server_negotiation.cc:407] Connection header received
0926 20:45:12.187723 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:12.187748 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:12.188196 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:12.188679 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:12.189288 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:12.190610 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:12.192012 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:12.192890 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:12.193125 (+ 235us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:12.194621 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:12.194630 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:12.194646 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:12.195216 (+ 570us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:12.195295 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:12.195326 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:12.195509 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:12.195784 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 20:45:12.195999 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1059,"thread_start_us":560,"threads_started":1}
I20260926 20:45:12.199283 12885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:12.200511 12885 log.cc:824] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:12.201113 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:12.203495 12885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551: No bootstrap required, opened a new log
I20260926 20:45:12.212165 12885 raft_consensus.cc:348] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24c0526d4ee4ccdbb34515f354e0551" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34719 } }
I20260926 20:45:12.212662 12885 raft_consensus.cc:374] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:12.212787 12885 raft_consensus.cc:729] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d24c0526d4ee4ccdbb34515f354e0551, State: Initialized, Role: FOLLOWER
I20260926 20:45:12.213467 12885 consensus_queue.cc:260] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24c0526d4ee4ccdbb34515f354e0551" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34719 } }
I20260926 20:45:12.213774 12885 raft_consensus.cc:388] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:12.214003 12888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:12.206086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:57927 (local address 127.0.0.1:34719)
0926 20:45:12.206388 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:12.206398 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:12.206552 (+ 154us) server_negotiation.cc:407] Connection header received
0926 20:45:12.206833 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:12.206837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:12.206923 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:12.207066 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:12.207653 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:12.208681 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:12.210896 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:12.211501 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:12.211672 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:12.213037 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:12.213042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:12.213046 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:12.213067 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:12.213130 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:12.213147 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:12.213296 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:12.213511 (+ 215us) server_negotiation.cc:299] Negotiation successful
0926 20:45:12.213571 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260926 20:45:12.213981 12885 raft_consensus.cc:482] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:12.214249 12885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:12.216907 12885 raft_consensus.cc:504] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24c0526d4ee4ccdbb34515f354e0551" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34719 } }
I20260926 20:45:12.217415 12885 leader_election.cc:302] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d24c0526d4ee4ccdbb34515f354e0551; no voters:
I20260926 20:45:12.217979 12885 leader_election.cc:288] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:12.218256 12895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:12.219105 12895 raft_consensus.cc:686] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [term 1 LEADER]: Becoming Leader. State: Replica: d24c0526d4ee4ccdbb34515f354e0551, State: Running, Role: LEADER
I20260926 20:45:12.219713 12895 consensus_queue.cc:237] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24c0526d4ee4ccdbb34515f354e0551" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34719 } }
I20260926 20:45:12.220314 12885 sys_catalog.cc:564] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:12.222692 12896 sys_catalog.cc:455] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d24c0526d4ee4ccdbb34515f354e0551" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24c0526d4ee4ccdbb34515f354e0551" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34719 } } }
I20260926 20:45:12.223155 12896 sys_catalog.cc:458] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:12.224136 12897 sys_catalog.cc:455] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d24c0526d4ee4ccdbb34515f354e0551. Latest consensus state: current_term: 1 leader_uuid: "d24c0526d4ee4ccdbb34515f354e0551" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d24c0526d4ee4ccdbb34515f354e0551" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34719 } } }
I20260926 20:45:12.224429 12897 sys_catalog.cc:458] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:12.225760 12900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:12.230993 12900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:12.241891 12900 catalog_manager.cc:1347] Generated new cluster ID: 1bcb97bf493d4a849e00fe77b044c424
I20260926 20:45:12.242017 12900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:12.285174 12900 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:12.286667 12900 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:12.303604 12900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d24c0526d4ee4ccdbb34515f354e0551: Generated new TSK 0
I20260926 20:45:12.304857 12900 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:12.312335 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12818
W20260926 20:45:12.313738 12880 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.115:58245
W20260926 20:45:12.314633 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12818: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:45:12.314778 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 12818
W20260926 20:45:12.332048 12803 connection.cc:535] client connection to 127.0.0.1:34719 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34719): Connection reset by peer (error 104)
2026-09-26T20:45:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2085 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:12 dist-test-slave-4d5t krb5kdc[12916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:12 dist-test-slave-4d5t krb5kdc[12916](info): set up 2 sockets
Sep 26 20:45:12 dist-test-slave-4d5t krb5kdc[12916](info): commencing operation
krb5kdc: starting...
W20260926 20:45:13.494207 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.004s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:13 dist-test-slave-4d5t krb5kdc[12916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 20:45:13.784775 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45271
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:14.141382 12929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:14.141713 12929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:14.141889 12929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:14.149372 12929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:14.149497 12929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:14.149588 12929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:14.149703 12929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:14.149767 12929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:14.160111 12929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45271
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:14.161505 12929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:14.162853 12929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:14.176401 12937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:14.176542 12934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:14.176532 12935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:14.178069 12929 server_base.cc:1034] running on GCE node
I20260926 20:45:14.178910 12929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:14.180461 12929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:14.181730 12929 hybrid_clock.cc:648] HybridClock initialized: now 1790455514181689 us; error 37 us; skew 500 ppm
I20260926 20:45:14.182165 12929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:14 dist-test-slave-4d5t krb5kdc[12916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455514, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:14.185947 12929 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 20:45:14.187742 12929 webserver.cc:460] Webserver started at http://127.0.0.1:34037/ using document root <none> and password file <none>
I20260926 20:45:14.188467 12929 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:14.188668 12929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:14.189006 12929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:14.192368 12929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "72d718c9ca74425d889ff8b3f6d6c0b1"
format_stamp: "Formatted at 2026-09-26 20:45:14 on dist-test-slave-4d5t"
server_key: "dfd2ee43b94ea34423b5e677f756ad89"
server_key_iv: "fc95567bd7dd0c5e015b80c0f030dfd1"
server_key_version: "encryptionkey@0"
I20260926 20:45:14.193275 12929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "72d718c9ca74425d889ff8b3f6d6c0b1"
format_stamp: "Formatted at 2026-09-26 20:45:14 on dist-test-slave-4d5t"
server_key: "dfd2ee43b94ea34423b5e677f756ad89"
server_key_iv: "fc95567bd7dd0c5e015b80c0f030dfd1"
server_key_version: "encryptionkey@0"
I20260926 20:45:14.199682 12929 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260926 20:45:14.203960 12944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:14.205734 12929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:45:14.205974 12929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "72d718c9ca74425d889ff8b3f6d6c0b1"
format_stamp: "Formatted at 2026-09-26 20:45:14 on dist-test-slave-4d5t"
server_key: "dfd2ee43b94ea34423b5e677f756ad89"
server_key_iv: "fc95567bd7dd0c5e015b80c0f030dfd1"
server_key_version: "encryptionkey@0"
I20260926 20:45:14.206228 12929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:14.213238 12929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:14.218775 12929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:14.219151 12929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:14.239442 12929 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45271
I20260926 20:45:14.239485 12995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45271 every 8 connection(s)
I20260926 20:45:14.240948 12929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:14.243203 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 12929
I20260926 20:45:14.243403 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:14.244009 23731 external_mini_cluster.cc:1428] Setting key f5f8c4699364896e099fcc5ddd7c87a3
I20260926 20:45:14.245965 12996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:14.252410 12999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:14.246020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:47181 (local address 127.0.0.1:45271)
0926 20:45:14.246886 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:14.246897 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:14.246939 (+ 42us) server_negotiation.cc:407] Connection header received
0926 20:45:14.248665 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:14.248686 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:14.249030 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:14.249379 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:14.250793 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:14.250800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:14.250813 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:14.250828 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:14.250912 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:14.250936 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:14.251122 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:14.251226 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 20:45:14.251439 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":381,"threads_started":1}
I20260926 20:45:14.256198 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:14.259298 12996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1: Bootstrap starting.
I20260926 20:45:14.263499 12996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:14.264470 12999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:14.261024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:42015 (local address 127.0.0.1:45271)
0926 20:45:14.261399 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:14.261415 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:14.261959 (+ 544us) server_negotiation.cc:407] Connection header received
0926 20:45:14.262326 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:14.262335 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:14.262411 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:14.262594 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:14.263689 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:14.263693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:14.263697 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:14.263721 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:14.263768 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:14.263786 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:14.263944 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:14.264092 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 20:45:14.264172 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 20:45:14.264910 12996 log.cc:824] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:14.268196 12996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1: No bootstrap required, opened a new log
I20260926 20:45:14.276253 12996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72d718c9ca74425d889ff8b3f6d6c0b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45271 } }
I20260926 20:45:14.276806 12996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:14.276926 12996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72d718c9ca74425d889ff8b3f6d6c0b1, State: Initialized, Role: FOLLOWER
I20260926 20:45:14.277555 12996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72d718c9ca74425d889ff8b3f6d6c0b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45271 } }
I20260926 20:45:14.277807 12996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:14.277966 12996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:14.278159 12996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:14.280406 12996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72d718c9ca74425d889ff8b3f6d6c0b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45271 } }
I20260926 20:45:14.280977 12996 leader_election.cc:302] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72d718c9ca74425d889ff8b3f6d6c0b1; no voters:
I20260926 20:45:14.281639 12996 leader_election.cc:288] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:14.281798 13006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:14.282508 13006 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [term 1 LEADER]: Becoming Leader. State: Replica: 72d718c9ca74425d889ff8b3f6d6c0b1, State: Running, Role: LEADER
I20260926 20:45:14.283109 13006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72d718c9ca74425d889ff8b3f6d6c0b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45271 } }
I20260926 20:45:14.284211 12996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:14.286911 13007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72d718c9ca74425d889ff8b3f6d6c0b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72d718c9ca74425d889ff8b3f6d6c0b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45271 } } }
I20260926 20:45:14.287429 13007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:14.286906 13008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72d718c9ca74425d889ff8b3f6d6c0b1. Latest consensus state: current_term: 1 leader_uuid: "72d718c9ca74425d889ff8b3f6d6c0b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72d718c9ca74425d889ff8b3f6d6c0b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45271 } } }
I20260926 20:45:14.288386 13008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:14.291934 13012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:14.300355 13012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:14.311808 13012 catalog_manager.cc:1347] Generated new cluster ID: 56b92cf5e5314c3395a8c4ff795f6124
I20260926 20:45:14.311949 13012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:14.334132 13012 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:14.334611 13012 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:14.345715 13012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72d718c9ca74425d889ff8b3f6d6c0b1: Generated new TSK 0
I20260926 20:45:14.346961 13012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:14.364099 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12929
W20260926 20:45:14.365309 12991 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.115:47181
W20260926 20:45:14.366065 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 12929: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:45:14.366214 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 12929
2026-09-26T20:45:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2048 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:14 dist-test-slave-4d5t krb5kdc[13027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:14 dist-test-slave-4d5t krb5kdc[13027](info): set up 2 sockets
Sep 26 20:45:14 dist-test-slave-4d5t krb5kdc[13027](info): commencing operation
krb5kdc: starting...
W20260926 20:45:15.550140 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:15 dist-test-slave-4d5t krb5kdc[13027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:45:15.836710 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:46219
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:46219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:16.173743 13040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:16.174054 13040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:16.174225 13040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:16.182044 13040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:16.182161 13040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:16.182240 13040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:16.182353 13040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:16.182415 13040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:16.192565 13040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:46219
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:46219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:16.193903 13040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:16.195221 13040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:16.207507 13045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:16.207470 13046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:16.207474 13048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:16.208526 13040 server_base.cc:1034] running on GCE node
I20260926 20:45:16.209336 13040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:16.210760 13040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:16.212009 13040 hybrid_clock.cc:648] HybridClock initialized: now 1790455516211965 us; error 39 us; skew 500 ppm
I20260926 20:45:16.212378 13040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:16 dist-test-slave-4d5t krb5kdc[13027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:16.215569 13040 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:45:16.217211 13040 webserver.cc:460] Webserver started at http://127.23.44.254:34639/ using document root <none> and password file <none>
I20260926 20:45:16.217926 13040 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:16.218071 13040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:16.218374 13040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:16.221550 13040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "a0959d42fc6d46d4a6050c9ecdcf1b8e"
format_stamp: "Formatted at 2026-09-26 20:45:16 on dist-test-slave-4d5t"
server_key: "d7ed7e3b8bb4cd909a46ec9ce8867c7d"
server_key_iv: "a817799353e886d267a174f7838492ab"
server_key_version: "encryptionkey@0"
I20260926 20:45:16.222384 13040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "a0959d42fc6d46d4a6050c9ecdcf1b8e"
format_stamp: "Formatted at 2026-09-26 20:45:16 on dist-test-slave-4d5t"
server_key: "d7ed7e3b8bb4cd909a46ec9ce8867c7d"
server_key_iv: "a817799353e886d267a174f7838492ab"
server_key_version: "encryptionkey@0"
I20260926 20:45:16.228320 13040 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260926 20:45:16.232527 13055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:16.234372 13040 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:45:16.234608 13040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "a0959d42fc6d46d4a6050c9ecdcf1b8e"
format_stamp: "Formatted at 2026-09-26 20:45:16 on dist-test-slave-4d5t"
server_key: "d7ed7e3b8bb4cd909a46ec9ce8867c7d"
server_key_iv: "a817799353e886d267a174f7838492ab"
server_key_version: "encryptionkey@0"
I20260926 20:45:16.234839 13040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:16.262476 13040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:16.268013 13040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:16.268354 13040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:16.289194 13040 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:46219
I20260926 20:45:16.289244 13106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:46219 every 8 connection(s)
I20260926 20:45:16.290614 13040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:16.294502 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 13040
I20260926 20:45:16.294706 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:16.294710 13107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:16.295264 23731 external_mini_cluster.cc:1428] Setting key fdc75411a19ee7bab06cc6b6c2ac5657
I20260926 20:45:16.308276 13107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e: Bootstrap starting.
Sep 26 20:45:16 dist-test-slave-4d5t krb5kdc[13027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:45:16.311712 13107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:16.312831 13107 log.cc:824] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:16.315802 13107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e: No bootstrap required, opened a new log
I20260926 20:45:16.321489 13110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:16.297718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47387 (local address 127.23.44.254:46219)
0926 20:45:16.298618 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:16.298631 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:16.298679 (+ 48us) server_negotiation.cc:407] Connection header received
0926 20:45:16.301832 (+ 3153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:16.301862 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:16.302309 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:16.302753 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:16.304228 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:16.306015 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:16.307324 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:16.308011 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:16.308214 (+ 203us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:16.313205 (+ 4991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:16.313217 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:16.313238 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:16.313303 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:16.316536 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:16.317557 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:16.317568 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:16.317574 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:16.317754 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:16.318407 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:16.318413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:16.318415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:16.319253 (+ 838us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:16.319529 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:16.320047 (+ 518us) server_negotiation.cc:299] Negotiation successful
0926 20:45:16.320377 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":420,"threads_started":1}
I20260926 20:45:16.324868 13107 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0959d42fc6d46d4a6050c9ecdcf1b8e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 46219 } }
I20260926 20:45:16.325242 13107 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:16.325366 13107 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0959d42fc6d46d4a6050c9ecdcf1b8e, State: Initialized, Role: FOLLOWER
I20260926 20:45:16.325398 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:16.325946 13107 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0959d42fc6d46d4a6050c9ecdcf1b8e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 46219 } }
I20260926 20:45:16.326195 13107 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:16.326331 13107 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:16.326520 13107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:16.328742 13107 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0959d42fc6d46d4a6050c9ecdcf1b8e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 46219 } }
I20260926 20:45:16.329326 13107 leader_election.cc:302] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0959d42fc6d46d4a6050c9ecdcf1b8e; no voters:
I20260926 20:45:16.329974 13107 leader_election.cc:288] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:16.330099 13116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:16.332214 13116 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [term 1 LEADER]: Becoming Leader. State: Replica: a0959d42fc6d46d4a6050c9ecdcf1b8e, State: Running, Role: LEADER
I20260926 20:45:16.332300 13107 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:16.332883 13116 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0959d42fc6d46d4a6050c9ecdcf1b8e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 46219 } }
I20260926 20:45:16.335974 13117 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0959d42fc6d46d4a6050c9ecdcf1b8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0959d42fc6d46d4a6050c9ecdcf1b8e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 46219 } } }
I20260926 20:45:16.336503 13118 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0959d42fc6d46d4a6050c9ecdcf1b8e. Latest consensus state: current_term: 1 leader_uuid: "a0959d42fc6d46d4a6050c9ecdcf1b8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0959d42fc6d46d4a6050c9ecdcf1b8e" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 46219 } } }
I20260926 20:45:16.336875 13118 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:16.337563 13117 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:16.345064 13124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:16.352584 13124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:16.353564 13110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:16.331421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56125 (local address 127.23.44.254:46219)
0926 20:45:16.331640 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:16.331649 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:16.331676 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:45:16.332043 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:16.332050 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:16.332133 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:16.332219 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:16.334524 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:16.336412 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:16.339119 (+ 2707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:16.340086 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:16.340220 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:16.344696 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:16.344703 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:16.344707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:16.344748 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:16.349839 (+ 5091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:16.350983 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:16.350990 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:16.350994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:16.351145 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:16.351663 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:16.351675 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:16.351678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:16.352120 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:16.352411 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:16.352956 (+ 545us) server_negotiation.cc:299] Negotiation successful
0926 20:45:16.353150 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:45:16.367301 13124 catalog_manager.cc:1347] Generated new cluster ID: b3b6f77d78e048678740c99a1fef7450
I20260926 20:45:16.367481 13124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:16.376227 13124 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:16.377722 13124 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:16.385174 13124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0959d42fc6d46d4a6050c9ecdcf1b8e: Generated new TSK 0
I20260926 20:45:16.386137 13124 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:16.392324 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 13040
I20260926 20:45:16.393615 13102 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:16.485635 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 13040
W20260926 20:45:16.505384 13025 connection.cc:535] client connection to 127.23.44.254:46219 recv error: Network error: failed to read from TLS socket (remote: 127.23.44.254:46219): Connection reset by peer (error 104)
2026-09-26T20:45:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2123 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:16 dist-test-slave-4d5t krb5kdc[13139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:16 dist-test-slave-4d5t krb5kdc[13139](info): set up 2 sockets
Sep 26 20:45:16 dist-test-slave-4d5t krb5kdc[13139](info): commencing operation
krb5kdc: starting...
W20260926 20:45:17.617450 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.006s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:17 dist-test-slave-4d5t krb5kdc[13139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:45:17.904892 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:37045
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:34967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:37045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:18.243664 13152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:18.243969 13152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:18.244141 13152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:18.251638 13152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:18.251761 13152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:18.251858 13152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:18.251945 13152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:18.251986 13152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:18.262187 13152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:34967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:37045
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:37045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:18.263558 13152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:18.264977 13152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:18.277133 13158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:18.277181 13160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:18.277453 13157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:18.278258 13152 server_base.cc:1034] running on GCE node
I20260926 20:45:18.279026 13152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:18.280465 13152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:18.281719 13152 hybrid_clock.cc:648] HybridClock initialized: now 1790455518281668 us; error 44 us; skew 500 ppm
I20260926 20:45:18.282083 13152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:18 dist-test-slave-4d5t krb5kdc[13139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455518, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:18.285320 13152 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:45:18.286983 13152 webserver.cc:460] Webserver started at http://127.23.44.254:45061/ using document root <none> and password file <none>
I20260926 20:45:18.287736 13152 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:18.287876 13152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:18.288210 13152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:18.291384 13152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "5556ed06684a413fbf667bb70c4d3943"
format_stamp: "Formatted at 2026-09-26 20:45:18 on dist-test-slave-4d5t"
server_key: "ffaf427e65d61d1b005eb259ee0357e2"
server_key_iv: "bb9b650a40005ee1e38350fd24871ed0"
server_key_version: "encryptionkey@0"
I20260926 20:45:18.292215 13152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "5556ed06684a413fbf667bb70c4d3943"
format_stamp: "Formatted at 2026-09-26 20:45:18 on dist-test-slave-4d5t"
server_key: "ffaf427e65d61d1b005eb259ee0357e2"
server_key_iv: "bb9b650a40005ee1e38350fd24871ed0"
server_key_version: "encryptionkey@0"
I20260926 20:45:18.297876 13152 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 20:45:18.302484 13167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:18.304095 13152 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 20:45:18.304313 13152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "5556ed06684a413fbf667bb70c4d3943"
format_stamp: "Formatted at 2026-09-26 20:45:18 on dist-test-slave-4d5t"
server_key: "ffaf427e65d61d1b005eb259ee0357e2"
server_key_iv: "bb9b650a40005ee1e38350fd24871ed0"
server_key_version: "encryptionkey@0"
I20260926 20:45:18.304584 13152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:18.335453 13152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:18.339942 13152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:18.340283 13152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:18.360541 13152 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:37045
I20260926 20:45:18.360601 13218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:37045 every 8 connection(s)
I20260926 20:45:18.362071 13152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:18.363641 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 13152
I20260926 20:45:18.363822 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:18.364337 23731 external_mini_cluster.cc:1428] Setting key d58568544ffc37312a749873c4297dc8
I20260926 20:45:18.366932 13219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:18.378657 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:18.366297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51529 (local address 127.23.44.254:37045)
0926 20:45:18.367208 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:18.367223 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:18.367264 (+ 41us) server_negotiation.cc:407] Connection header received
0926 20:45:18.369016 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:18.369039 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:18.369441 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:18.369782 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:18.370394 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:18.371727 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:18.373675 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:18.374815 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:18.374955 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:18.376290 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:18.376298 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:18.376317 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:18.376977 (+ 660us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:18.377059 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:18.377092 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:18.377234 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:18.377511 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 20:45:18.377664 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":404,"threads_started":1}
I20260926 20:45:18.380012 13219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943: Bootstrap starting.
I20260926 20:45:18.382622 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:18.383414 13219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:18.384402 13219 log.cc:824] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:18.388163 13219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943: No bootstrap required, opened a new log
I20260926 20:45:18.393530 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:18.387087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54863 (local address 127.23.44.254:37045)
0926 20:45:18.387285 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:18.387294 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:18.387705 (+ 411us) server_negotiation.cc:407] Connection header received
0926 20:45:18.387929 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:18.387934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:18.387994 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:18.388107 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:18.388719 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:18.389478 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:18.390816 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:18.391294 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:18.391467 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:18.392764 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:18.392769 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:18.392772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:18.392793 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:18.392830 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:18.392845 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:18.392981 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:18.393181 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 20:45:18.393256 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 20:45:18.397208 13219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5556ed06684a413fbf667bb70c4d3943" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 37045 } }
I20260926 20:45:18.397636 13219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:18.397758 13219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5556ed06684a413fbf667bb70c4d3943, State: Initialized, Role: FOLLOWER
I20260926 20:45:18.398367 13219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5556ed06684a413fbf667bb70c4d3943" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 37045 } }
I20260926 20:45:18.398645 13219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:18.398813 13219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:18.399011 13219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:18.402060 13219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5556ed06684a413fbf667bb70c4d3943" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 37045 } }
I20260926 20:45:18.402633 13219 leader_election.cc:302] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5556ed06684a413fbf667bb70c4d3943; no voters:
I20260926 20:45:18.403275 13219 leader_election.cc:288] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:18.403437 13229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:18.404227 13229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [term 1 LEADER]: Becoming Leader. State: Replica: 5556ed06684a413fbf667bb70c4d3943, State: Running, Role: LEADER
I20260926 20:45:18.404826 13229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5556ed06684a413fbf667bb70c4d3943" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 37045 } }
I20260926 20:45:18.405520 13219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:18.407895 13231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5556ed06684a413fbf667bb70c4d3943. Latest consensus state: current_term: 1 leader_uuid: "5556ed06684a413fbf667bb70c4d3943" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5556ed06684a413fbf667bb70c4d3943" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 37045 } } }
I20260926 20:45:18.409132 13231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:18.409081 13230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5556ed06684a413fbf667bb70c4d3943" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5556ed06684a413fbf667bb70c4d3943" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 37045 } } }
I20260926 20:45:18.411201 13230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:18.411227 13236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:18.417183 13236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:18.427697 13236 catalog_manager.cc:1347] Generated new cluster ID: 13553087c61d44a5a13a63037205cc90
I20260926 20:45:18.427834 13236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:18.444205 13236 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:18.445652 13236 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:18.460784 13236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5556ed06684a413fbf667bb70c4d3943: Generated new TSK 0
I20260926 20:45:18.461933 13236 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:18.477837 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 13152
W20260926 20:45:18.479113 13214 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51529
W20260926 20:45:18.479930 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 13152: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:45:18.480093 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 13152
W20260926 20:45:18.497601 13137 connection.cc:535] client connection to 127.23.44.254:37045 recv error: Network error: failed to read from TLS socket (remote: 127.23.44.254:37045): Connection reset by peer (error 104)
2026-09-26T20:45:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1992 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:18 dist-test-slave-4d5t krb5kdc[13250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:18 dist-test-slave-4d5t krb5kdc[13250](info): set up 2 sockets
Sep 26 20:45:18 dist-test-slave-4d5t krb5kdc[13250](info): commencing operation
krb5kdc: starting...
W20260926 20:45:19.629392 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:19 dist-test-slave-4d5t krb5kdc[13250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:45:19.912487 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:38085
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:45881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:38085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:20.266957 13263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:20.267284 13263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:20.267383 13263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:20.274907 13263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:20.275045 13263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:20.275095 13263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:20.275151 13263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:20.275202 13263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:20.285533 13263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:45881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:38085
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:38085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:20.286825 13263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:20.288305 13263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:20.302386 13271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:20.302443 13268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:20.302464 13269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:20.303701 13263 server_base.cc:1034] running on GCE node
I20260926 20:45:20.304512 13263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:20.306353 13263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:20.307632 13263 hybrid_clock.cc:648] HybridClock initialized: now 1790455520307597 us; error 60 us; skew 500 ppm
I20260926 20:45:20.308045 13263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:20 dist-test-slave-4d5t krb5kdc[13250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455520, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:20.316877 13263 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:45:20.318877 13263 webserver.cc:460] Webserver started at http://127.23.44.254:36247/ using document root <none> and password file <none>
I20260926 20:45:20.319726 13263 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:20.319901 13263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:20.320247 13263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:20.323441 13263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "df30a008bb4f4dc78c5fa6a063153153"
format_stamp: "Formatted at 2026-09-26 20:45:20 on dist-test-slave-4d5t"
server_key: "1d1a86a040c3ec9ad578382b55e414be"
server_key_iv: "d0b8f9cf5873680b0c83327f13361a7f"
server_key_version: "encryptionkey@0"
I20260926 20:45:20.324436 13263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "df30a008bb4f4dc78c5fa6a063153153"
format_stamp: "Formatted at 2026-09-26 20:45:20 on dist-test-slave-4d5t"
server_key: "1d1a86a040c3ec9ad578382b55e414be"
server_key_iv: "d0b8f9cf5873680b0c83327f13361a7f"
server_key_version: "encryptionkey@0"
I20260926 20:45:20.330106 13263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260926 20:45:20.333850 13278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:20.335404 13263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 20:45:20.335642 13263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "df30a008bb4f4dc78c5fa6a063153153"
format_stamp: "Formatted at 2026-09-26 20:45:20 on dist-test-slave-4d5t"
server_key: "1d1a86a040c3ec9ad578382b55e414be"
server_key_iv: "d0b8f9cf5873680b0c83327f13361a7f"
server_key_version: "encryptionkey@0"
I20260926 20:45:20.335888 13263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:20.342283 13263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:20.346829 13263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:20.347162 13263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:20.368039 13263 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:38085
I20260926 20:45:20.368080 13329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:38085 every 8 connection(s)
I20260926 20:45:20.369570 13263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:20.371107 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 13263
I20260926 20:45:20.371300 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:20.371824 23731 external_mini_cluster.cc:1428] Setting key 3730ac8a6ae9c6b0ff5212017fce3e94
I20260926 20:45:20.374568 13330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:20.380307 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:20.374164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55975 (local address 127.23.44.254:38085)
0926 20:45:20.375064 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:20.375076 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:20.375116 (+ 40us) server_negotiation.cc:407] Connection header received
0926 20:45:20.376802 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:20.376826 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:20.377188 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:20.377537 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:20.378649 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:20.378655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:20.378668 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:20.378682 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:20.378762 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:20.378785 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:20.378969 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:20.379089 (+ 120us) server_negotiation.cc:299] Negotiation successful
0926 20:45:20.379311 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":454,"threads_started":1}
I20260926 20:45:20.385066 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:20.387707 13330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153: Bootstrap starting.
I20260926 20:45:20.391624 13330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:20.392954 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:20.390046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44897 (local address 127.23.44.254:38085)
0926 20:45:20.390334 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:20.390342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:20.390594 (+ 252us) server_negotiation.cc:407] Connection header received
0926 20:45:20.390829 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:20.390839 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:20.390894 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:20.391013 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:20.392071 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:20.392080 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:20.392085 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:20.392114 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:20.392163 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:20.392180 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:20.392314 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:20.392605 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 20:45:20.392676 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260926 20:45:20.392911 13330 log.cc:824] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:20.395882 13330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153: No bootstrap required, opened a new log
I20260926 20:45:20.403437 13330 raft_consensus.cc:348] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df30a008bb4f4dc78c5fa6a063153153" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 38085 } }
I20260926 20:45:20.403913 13330 raft_consensus.cc:374] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:20.404035 13330 raft_consensus.cc:729] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df30a008bb4f4dc78c5fa6a063153153, State: Initialized, Role: FOLLOWER
I20260926 20:45:20.404680 13330 consensus_queue.cc:260] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df30a008bb4f4dc78c5fa6a063153153" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 38085 } }
I20260926 20:45:20.404934 13330 raft_consensus.cc:388] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:20.405093 13330 raft_consensus.cc:482] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:20.405274 13330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:20.407713 13330 raft_consensus.cc:504] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df30a008bb4f4dc78c5fa6a063153153" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 38085 } }
I20260926 20:45:20.408250 13330 leader_election.cc:302] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df30a008bb4f4dc78c5fa6a063153153; no voters:
I20260926 20:45:20.408924 13330 leader_election.cc:288] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:20.409070 13340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:20.409756 13340 raft_consensus.cc:686] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [term 1 LEADER]: Becoming Leader. State: Replica: df30a008bb4f4dc78c5fa6a063153153, State: Running, Role: LEADER
I20260926 20:45:20.410312 13340 consensus_queue.cc:237] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df30a008bb4f4dc78c5fa6a063153153" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 38085 } }
I20260926 20:45:20.411090 13330 sys_catalog.cc:564] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:20.413344 13341 sys_catalog.cc:455] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df30a008bb4f4dc78c5fa6a063153153" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df30a008bb4f4dc78c5fa6a063153153" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 38085 } } }
I20260926 20:45:20.413839 13341 sys_catalog.cc:458] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:20.415884 13347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:20.415788 13342 sys_catalog.cc:455] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df30a008bb4f4dc78c5fa6a063153153. Latest consensus state: current_term: 1 leader_uuid: "df30a008bb4f4dc78c5fa6a063153153" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df30a008bb4f4dc78c5fa6a063153153" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 38085 } } }
I20260926 20:45:20.416267 13342 sys_catalog.cc:458] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:20.421501 13347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:20.431543 13347 catalog_manager.cc:1347] Generated new cluster ID: 298afc978a9e427facbe6ae8b533e0fa
I20260926 20:45:20.431671 13347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:20.448076 13347 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:20.448622 13347 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:20.464057 13347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df30a008bb4f4dc78c5fa6a063153153: Generated new TSK 0
I20260926 20:45:20.465138 13347 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:20.470034 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 13263
W20260926 20:45:20.471201 13325 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55975
W20260926 20:45:20.471966 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 13263: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:45:20.472118 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 13263
2026-09-26T20:45:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1993 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:20 dist-test-slave-4d5t krb5kdc[13361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:20 dist-test-slave-4d5t krb5kdc[13361](info): set up 2 sockets
Sep 26 20:45:20 dist-test-slave-4d5t krb5kdc[13361](info): commencing operation
krb5kdc: starting...
W20260926 20:45:21.646276 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.004s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:21 dist-test-slave-4d5t krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:45:21.940778 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:40315
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:40731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:40315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:22.291844 13374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:22.292200 13374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:22.292377 13374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:22.300935 13374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:22.301098 13374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:22.301203 13374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:22.301311 13374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:22.301385 13374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:22.311887 13374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:40731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:40315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:40315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:22.313544 13374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:22.314879 13374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:22.328526 13379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:22.328629 13380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:22.328585 13382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:22.334494 13374 server_base.cc:1034] running on GCE node
I20260926 20:45:22.335443 13374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:22.337126 13374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:22.338439 13374 hybrid_clock.cc:648] HybridClock initialized: now 1790455522338367 us; error 45 us; skew 500 ppm
I20260926 20:45:22.338840 13374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:22 dist-test-slave-4d5t krb5kdc[13361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455522, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:22.342351 13374 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:45:22.344120 13374 webserver.cc:460] Webserver started at http://127.23.44.254:33237/ using document root <none> and password file <none>
I20260926 20:45:22.344946 13374 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:22.345068 13374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:22.345357 13374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:22.348776 13374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "d18446e4ec90408b8d00b69912168239"
format_stamp: "Formatted at 2026-09-26 20:45:22 on dist-test-slave-4d5t"
server_key: "c2d1501ed45e4d87ae4ac58e168402fe"
server_key_iv: "732dcd02f1a65aa02fd3ceb915ede7a1"
server_key_version: "encryptionkey@0"
I20260926 20:45:22.349622 13374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "d18446e4ec90408b8d00b69912168239"
format_stamp: "Formatted at 2026-09-26 20:45:22 on dist-test-slave-4d5t"
server_key: "c2d1501ed45e4d87ae4ac58e168402fe"
server_key_iv: "732dcd02f1a65aa02fd3ceb915ede7a1"
server_key_version: "encryptionkey@0"
I20260926 20:45:22.355412 13374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 20:45:22.359169 13389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:22.360772 13374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 20:45:22.360960 13374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "d18446e4ec90408b8d00b69912168239"
format_stamp: "Formatted at 2026-09-26 20:45:22 on dist-test-slave-4d5t"
server_key: "c2d1501ed45e4d87ae4ac58e168402fe"
server_key_iv: "732dcd02f1a65aa02fd3ceb915ede7a1"
server_key_version: "encryptionkey@0"
I20260926 20:45:22.361153 13374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:22.375509 13374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:22.381103 13374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:22.381479 13374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:22.404753 13374 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:40315
I20260926 20:45:22.404757 13440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:40315 every 8 connection(s)
I20260926 20:45:22.406237 13374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:22.409119 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 13374
I20260926 20:45:22.409308 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:22.409821 23731 external_mini_cluster.cc:1428] Setting key e8fb7a34fe7467ad8460efa43cae28d4
I20260926 20:45:22.411230 13441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 20:45:22 dist-test-slave-4d5t krb5kdc[13361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:45:22.424893 13441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239: Bootstrap starting.
I20260926 20:45:22.428848 13441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:22.430037 13441 log.cc:824] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:22.433382 13441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239: No bootstrap required, opened a new log
I20260926 20:45:22.433487 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:22.411672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57133 (local address 127.23.44.254:40315)
0926 20:45:22.412647 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:22.412663 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:22.412719 (+ 56us) server_negotiation.cc:407] Connection header received
0926 20:45:22.414770 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:22.414792 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:22.415239 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:22.415939 (+ 700us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:22.417554 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:22.418873 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:22.420140 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:22.420953 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:22.421167 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:22.425735 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:22.425744 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:22.425759 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:22.425803 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:22.429229 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:22.430011 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:22.430019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:22.430029 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:22.430262 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:22.430648 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:22.430654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:22.430657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:22.431226 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:22.431581 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:22.432007 (+ 426us) server_negotiation.cc:299] Negotiation successful
0926 20:45:22.432454 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":384,"threads_started":1}
I20260926 20:45:22.437433 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:22.441715 13441 raft_consensus.cc:348] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18446e4ec90408b8d00b69912168239" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40315 } }
I20260926 20:45:22.442261 13441 raft_consensus.cc:374] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:22.442471 13441 raft_consensus.cc:729] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d18446e4ec90408b8d00b69912168239, State: Initialized, Role: FOLLOWER
I20260926 20:45:22.443331 13441 consensus_queue.cc:260] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18446e4ec90408b8d00b69912168239" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40315 } }
I20260926 20:45:22.443621 13441 raft_consensus.cc:388] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:22.443765 13441 raft_consensus.cc:482] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:22.443979 13441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:22.446892 13441 raft_consensus.cc:504] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18446e4ec90408b8d00b69912168239" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40315 } }
I20260926 20:45:22.447420 13441 leader_election.cc:302] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d18446e4ec90408b8d00b69912168239; no voters:
I20260926 20:45:22.448112 13441 leader_election.cc:288] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:22.448248 13451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:22.450609 13441 sys_catalog.cc:564] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:22.452917 13451 raft_consensus.cc:686] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [term 1 LEADER]: Becoming Leader. State: Replica: d18446e4ec90408b8d00b69912168239, State: Running, Role: LEADER
I20260926 20:45:22.453634 13451 consensus_queue.cc:237] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18446e4ec90408b8d00b69912168239" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40315 } }
I20260926 20:45:22.457266 13453 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d18446e4ec90408b8d00b69912168239. Latest consensus state: current_term: 1 leader_uuid: "d18446e4ec90408b8d00b69912168239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18446e4ec90408b8d00b69912168239" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40315 } } }
I20260926 20:45:22.457702 13453 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:22.457388 13452 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d18446e4ec90408b8d00b69912168239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18446e4ec90408b8d00b69912168239" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40315 } } }
I20260926 20:45:22.457880 13452 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:22.462129 13457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:22.463091 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:22.442147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54927 (local address 127.23.44.254:40315)
0926 20:45:22.442376 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:22.442385 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:22.442825 (+ 440us) server_negotiation.cc:407] Connection header received
0926 20:45:22.443062 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:22.443071 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:22.443167 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:22.443270 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:22.445286 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:22.446406 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:22.447895 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:22.448767 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:22.449765 (+ 998us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:22.452178 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:22.452184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:22.452188 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:22.452235 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:22.459452 (+ 7217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:22.460233 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:22.460240 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:22.460244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:22.460415 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:22.460926 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:22.460933 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:22.460939 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:22.461399 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:22.462531 (+ 1132us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:22.462646 (+ 115us) server_negotiation.cc:299] Negotiation successful
0926 20:45:22.462806 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260926 20:45:22.469926 13457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:22.482155 13457 catalog_manager.cc:1347] Generated new cluster ID: fcbc4dd88908464a8cfb02690a900cf5
I20260926 20:45:22.482331 13457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:22.504503 13457 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:22.505939 13457 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:22.513367 13457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d18446e4ec90408b8d00b69912168239: Generated new TSK 0
I20260926 20:45:22.514477 13457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:22.533793 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 13374
I20260926 20:45:22.535459 13436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:22.631232 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 13374
2026-09-26T20:45:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2164 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:22 dist-test-slave-4d5t krb5kdc[13473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:22 dist-test-slave-4d5t krb5kdc[13473](info): set up 2 sockets
Sep 26 20:45:22 dist-test-slave-4d5t krb5kdc[13473](info): commencing operation
krb5kdc: starting...
W20260926 20:45:23.785441 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:23 dist-test-slave-4d5t krb5kdc[13473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:45:24.072633 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:44311
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:41811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:44311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:24.433111 13486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:24.433421 13486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:24.433514 13486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:24.441802 13486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:24.441932 13486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:24.441980 13486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:24.442044 13486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:24.442097 13486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:24.452567 13486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:41811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:44311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:44311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:24.453876 13486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:24.455160 13486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:24.467445 13494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:24.467605 13492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:24.467613 13491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:24.468771 13486 server_base.cc:1034] running on GCE node
I20260926 20:45:24.469700 13486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:24.471324 13486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:24.472615 13486 hybrid_clock.cc:648] HybridClock initialized: now 1790455524472519 us; error 44 us; skew 500 ppm
I20260926 20:45:24.473038 13486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:24 dist-test-slave-4d5t krb5kdc[13473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455524, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:24.476497 13486 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:45:24.478163 13486 webserver.cc:460] Webserver started at http://127.23.44.254:42247/ using document root <none> and password file <none>
I20260926 20:45:24.478929 13486 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:24.479279 13486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:24.479851 13486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:24.483987 13486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "f06469bec3784bf28d52867f4cd9e713"
format_stamp: "Formatted at 2026-09-26 20:45:24 on dist-test-slave-4d5t"
server_key: "2fd4f21029355b07c41592c44576e212"
server_key_iv: "8aff9cc22c1a69e7eafedac9cb94fd30"
server_key_version: "encryptionkey@0"
I20260926 20:45:24.485028 13486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "f06469bec3784bf28d52867f4cd9e713"
format_stamp: "Formatted at 2026-09-26 20:45:24 on dist-test-slave-4d5t"
server_key: "2fd4f21029355b07c41592c44576e212"
server_key_iv: "8aff9cc22c1a69e7eafedac9cb94fd30"
server_key_version: "encryptionkey@0"
I20260926 20:45:24.491945 13486 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 20:45:24.496624 13501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:24.498270 13486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:45:24.498512 13486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "f06469bec3784bf28d52867f4cd9e713"
format_stamp: "Formatted at 2026-09-26 20:45:24 on dist-test-slave-4d5t"
server_key: "2fd4f21029355b07c41592c44576e212"
server_key_iv: "8aff9cc22c1a69e7eafedac9cb94fd30"
server_key_version: "encryptionkey@0"
I20260926 20:45:24.498759 13486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:24.519784 13486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:24.524492 13486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:24.524835 13486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:24.546437 13486 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:44311
I20260926 20:45:24.546455 13552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:44311 every 8 connection(s)
I20260926 20:45:24.547930 13486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:24.551792 13553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:24.553202 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 13486
I20260926 20:45:24.553490 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:24.554160 23731 external_mini_cluster.cc:1428] Setting key 05fed83a031f712dee3fb8ee6f5cc838
I20260926 20:45:24.568174 13553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713: Bootstrap starting.
I20260926 20:45:24.569470 13556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:24.556095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54343 (local address 127.23.44.254:44311)
0926 20:45:24.557620 (+ 1525us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:24.557637 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:24.557685 (+ 48us) server_negotiation.cc:407] Connection header received
0926 20:45:24.559424 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:24.559450 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:24.559902 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:24.560274 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:24.560994 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:24.562280 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:24.564412 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:24.565302 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:24.565448 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:24.566914 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:24.566922 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:24.566936 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:24.567520 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:24.567612 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:24.567642 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:24.567733 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:24.567991 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 20:45:24.568314 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":553,"threads_started":1}
I20260926 20:45:24.572321 13553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:24.573505 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:24.573746 13553 log.cc:824] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:24.577456 13553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713: No bootstrap required, opened a new log
I20260926 20:45:24.585963 13556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:24.578541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59489 (local address 127.23.44.254:44311)
0926 20:45:24.578758 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:24.578765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:24.579202 (+ 437us) server_negotiation.cc:407] Connection header received
0926 20:45:24.579346 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:24.579354 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:24.579443 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:24.579568 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:24.580048 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:24.581387 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:24.582770 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:24.583612 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:24.583766 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:24.584883 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:24.584892 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:24.584903 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:24.584927 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:24.584975 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:24.584996 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:24.585107 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:24.585453 (+ 346us) server_negotiation.cc:299] Negotiation successful
0926 20:45:24.585556 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 20:45:24.588016 13553 raft_consensus.cc:348] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06469bec3784bf28d52867f4cd9e713" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44311 } }
I20260926 20:45:24.588410 13553 raft_consensus.cc:374] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:24.588486 13553 raft_consensus.cc:729] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06469bec3784bf28d52867f4cd9e713, State: Initialized, Role: FOLLOWER
I20260926 20:45:24.589120 13553 consensus_queue.cc:260] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06469bec3784bf28d52867f4cd9e713" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44311 } }
I20260926 20:45:24.589370 13553 raft_consensus.cc:388] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:24.589468 13553 raft_consensus.cc:482] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:24.589607 13553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:24.592377 13553 raft_consensus.cc:504] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06469bec3784bf28d52867f4cd9e713" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44311 } }
I20260926 20:45:24.593019 13553 leader_election.cc:302] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f06469bec3784bf28d52867f4cd9e713; no voters:
I20260926 20:45:24.593679 13553 leader_election.cc:288] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:24.593864 13563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:24.594750 13563 raft_consensus.cc:686] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [term 1 LEADER]: Becoming Leader. State: Replica: f06469bec3784bf28d52867f4cd9e713, State: Running, Role: LEADER
I20260926 20:45:24.595280 13563 consensus_queue.cc:237] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06469bec3784bf28d52867f4cd9e713" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44311 } }
I20260926 20:45:24.596127 13553 sys_catalog.cc:564] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:24.598440 13564 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f06469bec3784bf28d52867f4cd9e713" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06469bec3784bf28d52867f4cd9e713" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44311 } } }
I20260926 20:45:24.599084 13564 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:24.599221 13565 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f06469bec3784bf28d52867f4cd9e713. Latest consensus state: current_term: 1 leader_uuid: "f06469bec3784bf28d52867f4cd9e713" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06469bec3784bf28d52867f4cd9e713" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44311 } } }
I20260926 20:45:24.599594 13565 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:24.603305 13568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:24.611500 13568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:24.623637 13568 catalog_manager.cc:1347] Generated new cluster ID: 62d176d069044544923e6c81c4eb7984
I20260926 20:45:24.623822 13568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:24.642343 13568 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:24.644372 13568 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:24.664705 13568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f06469bec3784bf28d52867f4cd9e713: Generated new TSK 0
I20260926 20:45:24.665896 13568 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:24.680136 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 13486
W20260926 20:45:24.682040 13548 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54343
W20260926 20:45:24.683003 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 13486: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:45:24.683174 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 13486
2026-09-26T20:45:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2046 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:24 dist-test-slave-4d5t krb5kdc[13584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:24 dist-test-slave-4d5t krb5kdc[13584](info): set up 2 sockets
Sep 26 20:45:24 dist-test-slave-4d5t krb5kdc[13584](info): commencing operation
krb5kdc: starting...
W20260926 20:45:25.853459 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.009s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:26 dist-test-slave-4d5t krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:45:26.153666 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:38961
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:36211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:38961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:26.525019 13597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:26.525336 13597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:26.525514 13597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:26.533828 13597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:26.533984 13597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:26.534127 13597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:26.534228 13597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:26.534276 13597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:26.546327 13597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:36211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:38961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:38961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:26.547765 13597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:26.549515 13597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:26.563414 13603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:26.563452 13602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:26.563608 13605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:26.564352 13597 server_base.cc:1034] running on GCE node
I20260926 20:45:26.565279 13597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:26.566811 13597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:26.568034 13597 hybrid_clock.cc:648] HybridClock initialized: now 1790455526567983 us; error 47 us; skew 500 ppm
I20260926 20:45:26.568431 13597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:26 dist-test-slave-4d5t krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455526, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:26.578229 13597 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:45:26.580000 13597 webserver.cc:460] Webserver started at http://127.23.44.254:43959/ using document root <none> and password file <none>
I20260926 20:45:26.580806 13597 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:26.580955 13597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:26.581272 13597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:26.584432 13597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "cb5ca493f0e34bc6a53d6d0408880da9"
format_stamp: "Formatted at 2026-09-26 20:45:26 on dist-test-slave-4d5t"
server_key: "1be5280252dfaa95c3dca45223d34076"
server_key_iv: "5899145418591d0706387e4fbff9091b"
server_key_version: "encryptionkey@0"
I20260926 20:45:26.585311 13597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "cb5ca493f0e34bc6a53d6d0408880da9"
format_stamp: "Formatted at 2026-09-26 20:45:26 on dist-test-slave-4d5t"
server_key: "1be5280252dfaa95c3dca45223d34076"
server_key_iv: "5899145418591d0706387e4fbff9091b"
server_key_version: "encryptionkey@0"
I20260926 20:45:26.591279 13597 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260926 20:45:26.595407 13612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:26.597123 13597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 20:45:26.597498 13597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "cb5ca493f0e34bc6a53d6d0408880da9"
format_stamp: "Formatted at 2026-09-26 20:45:26 on dist-test-slave-4d5t"
server_key: "1be5280252dfaa95c3dca45223d34076"
server_key_iv: "5899145418591d0706387e4fbff9091b"
server_key_version: "encryptionkey@0"
I20260926 20:45:26.597932 13597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:26.604221 13597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:26.609416 13597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:26.609767 13597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:26.630393 13597 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:38961
I20260926 20:45:26.630607 13663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:38961 every 8 connection(s)
I20260926 20:45:26.631886 13597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:26.633067 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 13597
I20260926 20:45:26.633250 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:26.633684 23731 external_mini_cluster.cc:1428] Setting key 31cf022878f580bfe9f68e7809f96a5c
I20260926 20:45:26.636807 13664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:26.643204 13667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:26.635604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59399 (local address 127.23.44.254:38961)
0926 20:45:26.636824 (+ 1220us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:26.636842 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:26.637058 (+ 216us) server_negotiation.cc:407] Connection header received
0926 20:45:26.639414 (+ 2356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:26.639441 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:26.639829 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:26.640299 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:26.641439 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:26.641445 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:26.641457 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:26.641472 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:26.641544 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:26.641566 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:26.641883 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:26.642032 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 20:45:26.642231 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":410,"threads_started":1}
I20260926 20:45:26.647080 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:26.650532 13664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9: Bootstrap starting.
I20260926 20:45:26.654064 13667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:26.651514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58481 (local address 127.23.44.254:38961)
0926 20:45:26.651716 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:26.651724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:26.652062 (+ 338us) server_negotiation.cc:407] Connection header received
0926 20:45:26.652280 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:26.652284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:26.652337 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:26.652443 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:26.653404 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:26.653414 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:26.653417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 20:45:26.653432 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:26.653470 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 20:45:26.653485 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:26.653605 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:26.653740 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 20:45:26.653802 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 20:45:26.654362 13664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:26.655618 13664 log.cc:824] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:26.658546 13664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9: No bootstrap required, opened a new log
I20260926 20:45:26.666492 13664 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5ca493f0e34bc6a53d6d0408880da9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 38961 } }
I20260926 20:45:26.666903 13664 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 20:45:26.667021 13664 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb5ca493f0e34bc6a53d6d0408880da9, State: Initialized, Role: FOLLOWER
I20260926 20:45:26.667670 13664 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5ca493f0e34bc6a53d6d0408880da9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 38961 } }
I20260926 20:45:26.667906 13664 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 20:45:26.668077 13664 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 20:45:26.668248 13664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:26.670564 13664 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5ca493f0e34bc6a53d6d0408880da9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 38961 } }
I20260926 20:45:26.671118 13664 leader_election.cc:302] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb5ca493f0e34bc6a53d6d0408880da9; no voters:
I20260926 20:45:26.671834 13664 leader_election.cc:288] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 20:45:26.672031 13674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:26.673302 13674 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [term 1 LEADER]: Becoming Leader. State: Replica: cb5ca493f0e34bc6a53d6d0408880da9, State: Running, Role: LEADER
I20260926 20:45:26.673833 13674 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5ca493f0e34bc6a53d6d0408880da9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 38961 } }
I20260926 20:45:26.674052 13664 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:26.676717 13676 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb5ca493f0e34bc6a53d6d0408880da9. Latest consensus state: current_term: 1 leader_uuid: "cb5ca493f0e34bc6a53d6d0408880da9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5ca493f0e34bc6a53d6d0408880da9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 38961 } } }
I20260926 20:45:26.677184 13676 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:26.677690 13675 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb5ca493f0e34bc6a53d6d0408880da9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb5ca493f0e34bc6a53d6d0408880da9" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 38961 } } }
I20260926 20:45:26.677968 13675 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:26.678155 13683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:26.684448 13683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:26.694744 13683 catalog_manager.cc:1347] Generated new cluster ID: 0851fe5c48c047bdbc25036bb22b7f7e
I20260926 20:45:26.694893 13683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:26.709956 13683 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:26.710597 13683 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:26.723969 13683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb5ca493f0e34bc6a53d6d0408880da9: Generated new TSK 0
I20260926 20:45:26.725159 13683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:26.737746 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 13597
W20260926 20:45:26.738940 13659 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59399
W20260926 20:45:26.739749 23731 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 13597: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 20:45:26.739933 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 13597
2026-09-26T20:45:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2053 ms)
[----------] 18 tests from AuthTokenIssuingTest (37432 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:26 dist-test-slave-4d5t krb5kdc[13695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:26 dist-test-slave-4d5t krb5kdc[13695](info): set up 2 sockets
Sep 26 20:45:26 dist-test-slave-4d5t krb5kdc[13695](info): commencing operation
krb5kdc: starting...
W20260926 20:45:27.873608 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.006s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:28 dist-test-slave-4d5t krb5kdc[13695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:45:28.164513 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:40341
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:40341,127.23.44.253:35645,127.23.44.252:36635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:28.506520 13708 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 20:45:28.506865 13708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:28.506950 13708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:28.507033 13708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:28.514482 13708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:28.514592 13708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:28.514652 13708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:28.514698 13708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:28.514760 13708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:28.524838 13708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:40341,127.23.44.253:35645,127.23.44.252:36635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:40341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:28.526149 13708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:28.527441 13708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:28.541347 13713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:28.541373 13714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:28.542140 13716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:28.542649 13708 server_base.cc:1034] running on GCE node
I20260926 20:45:28.543534 13708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:28.545112 13708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:28.546396 13708 hybrid_clock.cc:648] HybridClock initialized: now 1790455528546309 us; error 59 us; skew 500 ppm
I20260926 20:45:28.546774 13708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:28 dist-test-slave-4d5t krb5kdc[13695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455528, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:28.550619 13708 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:45:28.552445 13708 webserver.cc:460] Webserver started at http://127.23.44.254:32951/ using document root <none> and password file <none>
I20260926 20:45:28.553236 13708 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:28.553392 13708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:28.553725 13708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:28.556967 13708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "48f6b9e010c54bf7baca350eb6133492"
format_stamp: "Formatted at 2026-09-26 20:45:28 on dist-test-slave-4d5t"
server_key: "751055e098991587a9d994b319b17f19"
server_key_iv: "4724ec63d7e81b5514c712cc96d92a51"
server_key_version: "encryptionkey@0"
I20260926 20:45:28.557818 13708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "48f6b9e010c54bf7baca350eb6133492"
format_stamp: "Formatted at 2026-09-26 20:45:28 on dist-test-slave-4d5t"
server_key: "751055e098991587a9d994b319b17f19"
server_key_iv: "4724ec63d7e81b5514c712cc96d92a51"
server_key_version: "encryptionkey@0"
I20260926 20:45:28.563374 13708 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:45:28.567204 13723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:28.568660 13708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 20:45:28.568871 13708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "48f6b9e010c54bf7baca350eb6133492"
format_stamp: "Formatted at 2026-09-26 20:45:28 on dist-test-slave-4d5t"
server_key: "751055e098991587a9d994b319b17f19"
server_key_iv: "4724ec63d7e81b5514c712cc96d92a51"
server_key_version: "encryptionkey@0"
I20260926 20:45:28.569128 13708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:28.606885 13708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:28.611251 13708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:28.611558 13708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:28.631290 13708 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:40341
I20260926 20:45:28.631310 13774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:40341 every 8 connection(s)
I20260926 20:45:28.632757 13708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:28.633908 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 13708
I20260926 20:45:28.634116 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:28.634783 23731 external_mini_cluster.cc:1428] Setting key 5f3a7fcab2b33fad83f3be99339b5533
I20260926 20:45:28.636824 13775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:28.643653 13775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.44.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.253@KRBTEST.COM" created.
Sep 26 20:45:28 dist-test-slave-4d5t krb5kdc[13695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455528, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:45:28.673574 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:28.647235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39060 (local address 127.23.44.254:40341)
0926 20:45:28.648194 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:28.648216 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:28.648771 (+ 555us) server_negotiation.cc:407] Connection header received
0926 20:45:28.650964 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:28.650971 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:28.651205 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:28.651438 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:28.652918 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:28.654249 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:28.655764 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:28.656400 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:28.656798 (+ 398us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:28.663983 (+ 7185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:28.664001 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:28.664025 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:28.664092 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:28.668402 (+ 4310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:28.669397 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:28.669408 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:28.669415 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:28.669622 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:28.670117 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:28.670125 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:28.670131 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:28.670910 (+ 779us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:28.671131 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:28.671732 (+ 601us) server_negotiation.cc:299] Negotiation successful
0926 20:45:28.672319 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":248,"server-negotiator.queue_time_us":687,"thread_start_us":446,"threads_started":1}
I20260926 20:45:28.673630 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:28.647235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:40341 (local address 127.0.0.1:39060)
0926 20:45:28.648532 (+ 1297us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:28.648631 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:28.650559 (+ 1928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:28.651449 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:28.651456 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:28.652232 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:28.652690 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:28.652711 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:28.654424 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:28.654430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:28.655609 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:28.655621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:28.656826 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:28.656833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:28.657017 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:28.657729 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:28.657761 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:28.663565 (+ 5804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:28.668707 (+ 5142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:28.668719 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:28.668730 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:28.669212 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:28.669763 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:28.669767 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:28.669770 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:28.669982 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:28.671124 (+ 1142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:28.671130 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:28.671312 (+ 182us) client_negotiation.cc:769] Sending connection context
0926 20:45:28.671538 (+ 226us) client_negotiation.cc:240] Negotiation successful
0926 20:45:28.672070 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"thread_start_us":829,"threads_started":1}
I20260926 20:45:28.676764 13775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:45:28.678426 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:28.677516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35645 (local address 127.0.0.1:52428)
0926 20:45:28.677882 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:28.678097 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260926 20:45:28.678766 13726 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
I20260926 20:45:28.680657 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:28.680101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35645 (local address 127.0.0.1:52438)
0926 20:45:28.680322 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:28.680432 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260926 20:45:28.680882 13775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:35645: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.253.keytab.
Entry for principal kudu/127.23.44.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.253.keytab.
I20260926 20:45:28.702246 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.253:35645
--webserver_interface=127.23.44.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.253.keytab
--principal=kudu/127.23.44.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:40341,127.23.44.253:35645,127.23.44.252:36635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:45:28.730264 13775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } attempt: 1
I20260926 20:45:28.731678 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:28.731099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35645 (local address 127.0.0.1:52442)
0926 20:45:28.731339 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:28.731491 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 20:45:28.732837 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:28.732366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35645 (local address 127.0.0.1:52448)
0926 20:45:28.732544 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:28.732660 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 20:45:28.733119 13775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:35645: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
I20260926 20:45:28.801527 13775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } attempt: 2
I20260926 20:45:28.803136 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:28.802531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35645 (local address 127.0.0.1:52450)
0926 20:45:28.802836 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:28.802957 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260926 20:45:28.804286 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:28.803852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35645 (local address 127.0.0.1:52458)
0926 20:45:28.804016 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:28.804118 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260926 20:45:28.804533 13775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:35645: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
I20260926 20:45:28.895879 13775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } attempt: 3
I20260926 20:45:28.897173 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:28.896705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35645 (local address 127.0.0.1:52466)
0926 20:45:28.896901 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:28.897008 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 20:45:28.898319 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:28.897896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35645 (local address 127.0.0.1:52482)
0926 20:45:28.898066 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:28.898152 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260926 20:45:28.898578 13775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:35645: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
I20260926 20:45:29.041945 13775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } attempt: 4
I20260926 20:45:29.043601 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.042873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35645 (local address 127.0.0.1:52488)
0926 20:45:29.043272 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.043390 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196}
I20260926 20:45:29.044901 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.044388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35645 (local address 127.0.0.1:52500)
0926 20:45:29.044597 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.044717 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260926 20:45:29.045284 13775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:35645: Network error: Client connection negotiation failed: client connection to 127.23.44.253:35645: connect: Connection refused (error 111)
W20260926 20:45:29.069063 13782 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 20:45:29.069465 13782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:29.069613 13782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:29.069743 13782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:29.078445 13782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:29.078610 13782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:29.078722 13782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:29.078826 13782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:29.078961 13782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:29.091094 13782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:40341,127.23.44.253:35645,127.23.44.252:36635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.253.keytab
--principal=kudu/127.23.44.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.253:35645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.44.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:29.092589 13782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:29.093954 13782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:29.106777 13788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:29.106760 13787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:29.106945 13790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:29.108690 13782 server_base.cc:1034] running on GCE node
I20260926 20:45:29.109607 13782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:29.111205 13782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:29.112509 13782 hybrid_clock.cc:648] HybridClock initialized: now 1790455529112452 us; error 70 us; skew 500 ppm
I20260926 20:45:29.112958 13782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:29 dist-test-slave-4d5t krb5kdc[13695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455529, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:29.117421 13782 init.cc:377] Logged in from keytab as kudu/127.23.44.253@KRBTEST.COM (short username kudu)
I20260926 20:45:29.119890 13782 webserver.cc:460] Webserver started at http://127.23.44.253:41687/ using document root <none> and password file <none>
I20260926 20:45:29.120911 13782 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:29.121073 13782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:29.121511 13782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:29.125286 13782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/data/instance:
uuid: "c2df151e455b4473a493b878c8174f7e"
format_stamp: "Formatted at 2026-09-26 20:45:29 on dist-test-slave-4d5t"
server_key: "9a495349cd832d79c3f3c677bda6edf9"
server_key_iv: "4f4706b49ed29234a02acc8d34990b5c"
server_key_version: "encryptionkey@0"
I20260926 20:45:29.126179 13782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/wal/instance:
uuid: "c2df151e455b4473a493b878c8174f7e"
format_stamp: "Formatted at 2026-09-26 20:45:29 on dist-test-slave-4d5t"
server_key: "9a495349cd832d79c3f3c677bda6edf9"
server_key_iv: "4f4706b49ed29234a02acc8d34990b5c"
server_key_version: "encryptionkey@0"
I20260926 20:45:29.131908 13782 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 20:45:29.136312 13797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:29.138144 13782 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 20:45:29.138391 13782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/wal
uuid: "c2df151e455b4473a493b878c8174f7e"
format_stamp: "Formatted at 2026-09-26 20:45:29 on dist-test-slave-4d5t"
server_key: "9a495349cd832d79c3f3c677bda6edf9"
server_key_iv: "4f4706b49ed29234a02acc8d34990b5c"
server_key_version: "encryptionkey@0"
I20260926 20:45:29.138681 13782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:29.182137 13782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:29.187601 13782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:29.187933 13782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:29.208369 13782 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.253:35645
I20260926 20:45:29.208413 13848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.253:35645 every 8 connection(s)
I20260926 20:45:29.209913 13782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/data/info.pb
I20260926 20:45:29.213496 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 13782
I20260926 20:45:29.213651 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-1/wal/instance
I20260926 20:45:29.213951 13849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:29.214183 23731 external_mini_cluster.cc:1428] Setting key b0637963e7a90753e9d9ec5d978cc7d3
I20260926 20:45:29.222144 13849 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.44.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.252@KRBTEST.COM" created.
Sep 26 20:45:29 dist-test-slave-4d5t krb5kdc[13695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455529, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.253@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:45:29.251008 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.226092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39066 (local address 127.23.44.254:40341)
0926 20:45:29.226660 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.226667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.227852 (+ 1185us) server_negotiation.cc:407] Connection header received
0926 20:45:29.229985 (+ 2133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.229994 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.230094 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.230245 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.232263 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.233469 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.235680 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.236510 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.236723 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.243800 (+ 7077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.243809 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.243813 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.243892 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.247463 (+ 3571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.248426 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.248432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.248436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.248640 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.249080 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.249087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.249093 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.249435 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.249615 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.250378 (+ 763us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.250621 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":281,"threads_started":1}
I20260926 20:45:29.252334 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.226762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:40341 (local address 127.0.0.1:39066)
0926 20:45:29.227640 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.227701 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:29.229683 (+ 1982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:29.230540 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:29.230550 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:29.231315 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:29.231999 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.232020 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.233678 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.233683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.235420 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.235450 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.236817 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.236824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.237086 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.237983 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:29.238007 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:29.243229 (+ 5222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:29.247711 (+ 4482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.247722 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.247743 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.248271 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.248763 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.248768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.248772 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.248962 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.249581 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:29.249591 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:29.249995 (+ 404us) client_negotiation.cc:769] Sending connection context
0926 20:45:29.250290 (+ 295us) client_negotiation.cc:240] Negotiation successful
0926 20:45:29.250854 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":389,"threads_started":1}
I20260926 20:45:29.254341 13849 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:45:29 dist-test-slave-4d5t krb5kdc[13695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455529, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.253@KRBTEST.COM for kudu/127.23.44.253@KRBTEST.COM
I20260926 20:45:29.273818 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.255380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35645 (local address 127.0.0.1:52510)
0926 20:45:29.255627 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.255649 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:29.255765 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:29.256897 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:29.256904 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:29.257460 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:29.257694 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.257709 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.259321 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.259329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.260923 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.260941 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.262432 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.262441 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.262719 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.263565 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:29.263595 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:29.266592 (+ 2997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:29.270442 (+ 3850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.270447 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.270450 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.271025 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.271605 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.271615 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.271620 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.271904 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.272801 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:29.272806 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:29.273028 (+ 222us) client_negotiation.cc:769] Sending connection context
0926 20:45:29.273168 (+ 140us) client_negotiation.cc:240] Negotiation successful
0926 20:45:29.273436 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260926 20:45:29.274132 13855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.255535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52510 (local address 127.23.44.253:35645)
0926 20:45:29.256345 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.256352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.256380 (+ 28us) server_negotiation.cc:407] Connection header received
0926 20:45:29.256428 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.256435 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.256730 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.256861 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.257895 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.259168 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.261086 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.262181 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.262379 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.266942 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.266964 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.266982 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.267020 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.270226 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.271176 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.271183 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.271186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.271330 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.272088 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.272092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.272095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.272543 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.272899 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.273431 (+ 532us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.273714 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":490,"threads_started":1}
I20260926 20:45:29.276860 13849 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:45:29.278313 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.277697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39098)
0926 20:45:29.277856 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.277974 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260926 20:45:29.278630 13798 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.252.keytab.
Entry for principal kudu/127.23.44.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.252.keytab.
I20260926 20:45:29.280448 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.252:36635
--webserver_interface=127.23.44.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.252.keytab
--principal=kudu/127.23.44.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:40341,127.23.44.253:35645,127.23.44.252:36635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:45:29.280869 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.280313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39114)
0926 20:45:29.280540 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.280701 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260926 20:45:29.281118 13849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:36635: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
I20260926 20:45:29.330540 13849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } attempt: 1
I20260926 20:45:29.332000 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.331426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39116)
0926 20:45:29.331725 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.331848 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260926 20:45:29.333434 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.333064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39132)
0926 20:45:29.333235 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.333325 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260926 20:45:29.333715 13849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:36635: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
I20260926 20:45:29.344678 13775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } attempt: 5
Sep 26 20:45:29 dist-test-slave-4d5t krb5kdc[13695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455528, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for kudu/127.23.44.253@KRBTEST.COM
I20260926 20:45:29.358296 13855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.345541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52526 (local address 127.23.44.253:35645)
0926 20:45:29.345760 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.345766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.345787 (+ 21us) server_negotiation.cc:407] Connection header received
0926 20:45:29.345965 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.345971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.346060 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.346173 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.346949 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.347666 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.348869 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.349294 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.349423 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.352934 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.352939 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.352942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.352974 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.355548 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.356250 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.356257 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.356261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.356386 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.356844 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.356850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.356854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.357152 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.357315 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.357725 (+ 410us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.357962 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260926 20:45:29.358296 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.345437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35645 (local address 127.0.0.1:52526)
0926 20:45:29.345630 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.345663 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:29.345793 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:29.346208 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:29.346213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:29.346614 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:29.346807 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.346816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.347798 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.347805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.348725 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.348736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.349447 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.349452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.349683 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.350338 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:29.350355 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:29.352723 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:29.355717 (+ 2994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.355723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.355727 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.356130 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.356534 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.356540 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.356544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.356733 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.357273 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:29.357278 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:29.357452 (+ 174us) client_negotiation.cc:769] Sending connection context
0926 20:45:29.357614 (+ 162us) client_negotiation.cc:240] Negotiation successful
0926 20:45:29.357932 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 20:45:29.359591 13775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:45:29.360772 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.360296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39142)
0926 20:45:29.360469 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.360546 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 20:45:29.362186 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.361705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39156)
0926 20:45:29.361977 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.362064 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
W20260926 20:45:29.362413 13775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:36635: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
I20260926 20:45:29.401947 13849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } attempt: 2
I20260926 20:45:29.403249 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.402781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39160)
0926 20:45:29.403001 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.403090 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260926 20:45:29.404238 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.403901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39166)
0926 20:45:29.404048 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.404129 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 20:45:29.404465 13849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:36635: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
I20260926 20:45:29.413648 13775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } attempt: 1
I20260926 20:45:29.415064 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.414461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39182)
0926 20:45:29.414716 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.414835 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260926 20:45:29.416677 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.416175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39184)
0926 20:45:29.416389 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.416496 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260926 20:45:29.416982 13775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:36635: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
I20260926 20:45:29.485288 13775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } attempt: 2
I20260926 20:45:29.486726 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.486007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39188)
0926 20:45:29.486313 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.486457 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260926 20:45:29.488094 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.487620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39194)
0926 20:45:29.487819 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.487928 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260926 20:45:29.488369 13775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:36635: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
I20260926 20:45:29.495780 13849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } attempt: 3
I20260926 20:45:29.497120 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.496641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39200)
0926 20:45:29.496847 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.496948 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260926 20:45:29.498468 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.498007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39206)
0926 20:45:29.498258 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.498347 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
W20260926 20:45:29.498878 13849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:36635: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
I20260926 20:45:29.594651 13775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } attempt: 3
I20260926 20:45:29.595986 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.595403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39208)
0926 20:45:29.595648 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.595766 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260926 20:45:29.597255 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.596793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39222)
0926 20:45:29.597022 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.597116 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260926 20:45:29.597524 13775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:36635: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
I20260926 20:45:29.642144 13849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } attempt: 4
I20260926 20:45:29.643575 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.642943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39230)
0926 20:45:29.643172 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.643383 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 20:45:29.644791 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.644343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39238)
0926 20:45:29.644515 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.644671 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 20:45:29.645040 13849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:36635: Network error: Client connection negotiation failed: client connection to 127.23.44.252:36635: connect: Connection refused (error 111)
W20260926 20:45:29.650766 13857 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 20:45:29.651237 13857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:29.651368 13857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:29.651510 13857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:29.660002 13857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:29.660243 13857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:29.660356 13857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:29.660467 13857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:29.660622 13857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:29.674521 13857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:40341,127.23.44.253:35645,127.23.44.252:36635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.252.keytab
--principal=kudu/127.23.44.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.252:36635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.44.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:29.676162 13857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:29.677666 13857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:29.691339 13863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:29.691417 13865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:29.691339 13862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:29.692343 13857 server_base.cc:1034] running on GCE node
I20260926 20:45:29.693209 13857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:29.694713 13857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:29.696002 13857 hybrid_clock.cc:648] HybridClock initialized: now 1790455529695928 us; error 45 us; skew 500 ppm
I20260926 20:45:29.696419 13857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:29 dist-test-slave-4d5t krb5kdc[13695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455529, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:29.700232 13857 init.cc:377] Logged in from keytab as kudu/127.23.44.252@KRBTEST.COM (short username kudu)
I20260926 20:45:29.701967 13857 webserver.cc:460] Webserver started at http://127.23.44.252:39889/ using document root <none> and password file <none>
I20260926 20:45:29.702733 13857 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:29.702879 13857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:29.703222 13857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:29.706473 13857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/data/instance:
uuid: "a279ba4046e349f18c9c73385345a22a"
format_stamp: "Formatted at 2026-09-26 20:45:29 on dist-test-slave-4d5t"
server_key: "73f1a90089eaf404ef6593ff490a2bf3"
server_key_iv: "74ca51b642061cd0005c54807b754063"
server_key_version: "encryptionkey@0"
I20260926 20:45:29.707350 13857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/wal/instance:
uuid: "a279ba4046e349f18c9c73385345a22a"
format_stamp: "Formatted at 2026-09-26 20:45:29 on dist-test-slave-4d5t"
server_key: "73f1a90089eaf404ef6593ff490a2bf3"
server_key_iv: "74ca51b642061cd0005c54807b754063"
server_key_version: "encryptionkey@0"
I20260926 20:45:29.713195 13857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260926 20:45:29.717609 13872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:29.719228 13857 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 20:45:29.719455 13857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/wal
uuid: "a279ba4046e349f18c9c73385345a22a"
format_stamp: "Formatted at 2026-09-26 20:45:29 on dist-test-slave-4d5t"
server_key: "73f1a90089eaf404ef6593ff490a2bf3"
server_key_iv: "74ca51b642061cd0005c54807b754063"
server_key_version: "encryptionkey@0"
I20260926 20:45:29.719717 13857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:29.742377 13857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:29.746954 13857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:29.747262 13857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:29.769088 13857 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.252:36635
I20260926 20:45:29.769091 13923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.252:36635 every 8 connection(s)
I20260926 20:45:29.770821 13857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/data/info.pb
I20260926 20:45:29.774766 13775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } attempt: 4
I20260926 20:45:29.775074 13924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:29.780782 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 13857
I20260926 20:45:29.780995 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790455151607908-23731-0/minicluster-data/master-2/wal/instance
I20260926 20:45:29.781595 23731 external_mini_cluster.cc:1428] Setting key 59db832aa3c0de2ec54fb9d5632001d9
I20260926 20:45:29.787654 13924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:45:29 dist-test-slave-4d5t krb5kdc[13695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455528, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for kudu/127.23.44.252@KRBTEST.COM
I20260926 20:45:29.801019 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.775510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39248)
0926 20:45:29.775736 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.775759 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:29.775877 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:29.779977 (+ 4100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:29.779983 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:29.780500 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:29.780728 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.780738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.782826 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.782836 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.784846 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.784870 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.786875 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.786884 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.787166 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.787988 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:29.788021 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:29.791974 (+ 3953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:29.797849 (+ 5875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.797854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.797857 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.798470 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.799025 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.799032 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.799036 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.799195 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.799990 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:29.799995 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:29.800163 (+ 168us) client_negotiation.cc:769] Sending connection context
0926 20:45:29.800304 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 20:45:29.800626 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 20:45:29.802596 13926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.776531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39248 (local address 127.23.44.252:36635)
0926 20:45:29.777422 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.777434 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.777487 (+ 53us) server_negotiation.cc:407] Connection header received
0926 20:45:29.779122 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.779145 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.779570 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.780017 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.780912 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.782604 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.785115 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.786635 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.787693 (+ 1058us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.792651 (+ 4958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.792666 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.792696 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.792762 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.797686 (+ 4924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.798642 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.798650 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.798659 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.798896 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.799322 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.799327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.799337 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.799848 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.800060 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.800606 (+ 546us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.801121 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":381,"threads_started":1}
Sep 26 20:45:29 dist-test-slave-4d5t krb5kdc[13695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455529, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.252@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
Sep 26 20:45:29 dist-test-slave-4d5t krb5kdc[13695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.252@KRBTEST.COM
I20260926 20:45:29.811401 13775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492: Bootstrap starting.
I20260926 20:45:29.813030 13930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.793234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39078 (local address 127.23.44.254:40341)
0926 20:45:29.794667 (+ 1433us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.794674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.795803 (+ 1129us) server_negotiation.cc:407] Connection header received
0926 20:45:29.796022 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.796028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.796108 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.796223 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.797849 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.798890 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.801030 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.801756 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.801898 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.805815 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.805822 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.805826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.805866 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.809744 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.810616 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.810621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.810624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.810771 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.811221 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.811227 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.811231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.811608 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.811811 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.812265 (+ 454us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.812496 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1271,"thread_start_us":1116,"threads_started":1}
I20260926 20:45:29.813035 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.792577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:40341 (local address 127.0.0.1:39078)
0926 20:45:29.795614 (+ 3037us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.795666 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:29.795845 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:29.796316 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:29.796325 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:29.797243 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:29.797670 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.797683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.799116 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.799121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.800841 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.800853 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.801949 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.801955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.802143 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.802945 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:29.802964 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:29.805596 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:29.809910 (+ 4314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.809924 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.809931 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.810472 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.810892 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.810897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.810901 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.811090 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.811761 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:29.811770 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:29.811963 (+ 193us) client_negotiation.cc:769] Sending connection context
0926 20:45:29.812155 (+ 192us) client_negotiation.cc:240] Negotiation successful
0926 20:45:29.812496 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2762,"thread_start_us":2349,"threads_started":1}
I20260926 20:45:29.813637 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.782799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39264 (local address 127.23.44.252:36635)
0926 20:45:29.796120 (+ 13321us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.796125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.796164 (+ 39us) server_negotiation.cc:407] Connection header received
0926 20:45:29.796210 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.796214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.796272 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.796376 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.797853 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.798784 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.800714 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.801375 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.801511 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.806591 (+ 5080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.806600 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.806606 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.806648 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.809706 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.810479 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.810485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.810487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.810620 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.811087 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.811092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.811095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.811494 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.811789 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.812943 (+ 1154us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.813180 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13207,"thread_start_us":13079,"threads_started":1}
I20260926 20:45:29.814767 13924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:45:29.815876 13775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:29.817291 13775 log.cc:824] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:29.821056 13775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492: No bootstrap required, opened a new log
Sep 26 20:45:29 dist-test-slave-4d5t krb5kdc[13695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455529, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.252@KRBTEST.COM for kudu/127.23.44.253@KRBTEST.COM
Sep 26 20:45:29 dist-test-slave-4d5t krb5kdc[13695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:45:29.833775 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.815978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:35645 (local address 127.0.0.1:52540)
0926 20:45:29.816410 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.816443 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:29.816588 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:29.816975 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:29.816981 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:29.817552 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:29.817810 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.817829 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.819155 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.819161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.820158 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.820175 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.821601 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.821609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.821884 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.822699 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:29.822733 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:29.825794 (+ 3061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:29.830207 (+ 4413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.830215 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.830219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.830868 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.831373 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.831379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.831383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.831574 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.832326 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:29.832335 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:29.832704 (+ 369us) client_negotiation.cc:769] Sending connection context
0926 20:45:29.832910 (+ 206us) client_negotiation.cc:240] Negotiation successful
0926 20:45:29.833292 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300}
I20260926 20:45:29.834303 13855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.816406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52540 (local address 127.23.44.253:35645)
0926 20:45:29.816646 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.816653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.816671 (+ 18us) server_negotiation.cc:407] Connection header received
0926 20:45:29.816728 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.816732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.816806 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.816910 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.818084 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.818999 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.820336 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.821142 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.821307 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.826084 (+ 4777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.826090 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.826093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.826130 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.829926 (+ 3796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.831037 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.831045 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.831054 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.831229 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.831702 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.831709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.831713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.832114 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.832547 (+ 433us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.833053 (+ 506us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.833387 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 20:45:29.835623 13924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:45:29.836192 13775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } }
I20260926 20:45:29.836694 13775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48f6b9e010c54bf7baca350eb6133492, State: Initialized, Role: FOLLOWER
I20260926 20:45:29.836766 13930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.815963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39090 (local address 127.23.44.254:40341)
0926 20:45:29.816190 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.816198 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.816223 (+ 25us) server_negotiation.cc:407] Connection header received
0926 20:45:29.816288 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.816295 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.816415 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.816538 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.818571 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.819809 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.821316 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.821794 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.821937 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.827255 (+ 5318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.827263 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.827268 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.827316 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.831915 (+ 4599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.833156 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.833162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.833166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.833407 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.834143 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.834147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.834150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.834746 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.835111 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.836080 (+ 969us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.836366 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 20:45:29.837350 13775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } }
I20260926 20:45:29.838759 13932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } } }
I20260926 20:45:29.839226 13932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:29.840215 13775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [sys.catalog]: configured and running, proceeding with master startup.
Sep 26 20:45:29 dist-test-slave-4d5t krb5kdc[13695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455529, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.252@KRBTEST.COM for kudu/127.23.44.252@KRBTEST.COM
Sep 26 20:45:29 dist-test-slave-4d5t krb5kdc[13695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.253@KRBTEST.COM
I20260926 20:45:29.860029 13855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.839107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52550 (local address 127.23.44.253:35645)
0926 20:45:29.839278 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.839284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.839510 (+ 226us) server_negotiation.cc:407] Connection header received
0926 20:45:29.839752 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.839757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.839839 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.839909 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.841715 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.842603 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.844582 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.845323 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.845438 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.852864 (+ 7426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.852871 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.852877 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.852932 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.856464 (+ 3532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.857290 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.857295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.857299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.857450 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.857862 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.857869 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.857872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.858196 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.858895 (+ 699us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.859330 (+ 435us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.859633 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260926 20:45:29.864660 13729 tablet.cc:2369] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:45:29.865901 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:29.865916 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.836546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39280)
0926 20:45:29.836802 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.836824 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:29.836939 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:29.837691 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:29.837695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:29.838247 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:29.838450 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.838463 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.839758 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.839763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.841247 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.841263 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.842744 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.842758 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.843058 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.843889 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:29.843913 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:29.846979 (+ 3066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:29.851897 (+ 4918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.851904 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.851909 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.852488 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.853017 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.853023 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.853027 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.853191 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.864658 (+ 11467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:29.864665 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:29.864909 (+ 244us) client_negotiation.cc:769] Sending connection context
0926 20:45:29.865097 (+ 188us) client_negotiation.cc:240] Negotiation successful
0926 20:45:29.865516 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":51}
I20260926 20:45:29.866119 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.837272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39280 (local address 127.23.44.252:36635)
0926 20:45:29.837437 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.837441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.837461 (+ 20us) server_negotiation.cc:407] Connection header received
0926 20:45:29.837498 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.837501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.837568 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.837677 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.838611 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.839610 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.841434 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.842512 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.842747 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.847165 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.847172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.847177 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.847216 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.851671 (+ 4455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.852707 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.852713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.852719 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.852894 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.853315 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.853322 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.853327 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.854140 (+ 813us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.854335 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.865309 (+ 10974us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.865727 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":133,"server-negotiator.queue_time_us":43}
W20260926 20:45:29.867806 13943 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:45:29.867938 13943 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:45:29.868750 13750 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "48f6b9e010c54bf7baca350eb6133492"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39090
I20260926 20:45:29.869194 13750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 20:45:29.854346 13708 master_runner.cc:186] Error getting master registration for 127.23.44.253:35645: OK, instance_id { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" instance_seqno: 1790455529202522 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 20:45:29.869390 13750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:29.869585 13708 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:45:29.871630 13750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } }
I20260926 20:45:29.872720 13924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a: Bootstrap starting.
I20260926 20:45:29.873672 13750 leader_election.cc:288] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [CANDIDATE]: Term 1 election: Requested vote from peers c2df151e455b4473a493b878c8174f7e (127.23.44.253:35645), a279ba4046e349f18c9c73385345a22a (127.23.44.252:36635)
I20260926 20:45:29.876899 13924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:29.878377 13924 log.cc:824] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:29.883934 13924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a: No bootstrap required, opened a new log
I20260926 20:45:29.897081 13924 raft_consensus.cc:348] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } }
I20260926 20:45:29.897655 13924 raft_consensus.cc:729] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a279ba4046e349f18c9c73385345a22a, State: Initialized, Role: FOLLOWER
I20260926 20:45:29.901311 13924 consensus_queue.cc:260] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } }
I20260926 20:45:29.920079 13855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.897129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52552 (local address 127.23.44.253:35645)
0926 20:45:29.897347 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.897354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.902064 (+ 4710us) server_negotiation.cc:407] Connection header received
0926 20:45:29.902346 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.902352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.902431 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.902503 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.905141 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.906247 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.908152 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.908866 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.908991 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.912455 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.912464 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.912469 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.912516 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.916453 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.917545 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.917551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.917556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.917753 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.918243 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.918249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.918252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.918625 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.918869 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.919448 (+ 579us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.919705 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:45:29.926471 13930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.889736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.23.44.254:40341)
0926 20:45:29.889909 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.889915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.898478 (+ 8563us) server_negotiation.cc:407] Connection header received
0926 20:45:29.898663 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.898671 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.898756 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.898865 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.901627 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.902879 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.905578 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.906525 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.906650 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.913246 (+ 6596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.913255 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.913263 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.913320 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.919303 (+ 5983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.922806 (+ 3503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.922816 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.922822 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.923131 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.923716 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.923725 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.923731 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.924371 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.924769 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.925506 (+ 737us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.925883 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 20:45:29.932118 13953 sys_catalog.cc:455] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } } }
I20260926 20:45:29.933969 13924 sys_catalog.cc:564] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:29.934957 13953 sys_catalog.cc:458] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:29.934959 13899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "48f6b9e010c54bf7baca350eb6133492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a279ba4046e349f18c9c73385345a22a"
I20260926 20:45:29.935622 13899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:29.939452 13899 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48f6b9e010c54bf7baca350eb6133492 in term 1.
I20260926 20:45:29.940171 13724 leader_election.cc:302] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48f6b9e010c54bf7baca350eb6133492, a279ba4046e349f18c9c73385345a22a; no voters:
I20260926 20:45:29.940774 13932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:29.944213 13849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } attempt: 5
I20260926 20:45:29.948027 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.889708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39294 (local address 127.23.44.252:36635)
0926 20:45:29.889868 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.889875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.900800 (+ 10925us) server_negotiation.cc:407] Connection header received
0926 20:45:29.900990 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.900996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.901079 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.901178 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.914295 (+ 13117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.915689 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.927881 (+ 12192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.928911 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.929063 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.935071 (+ 6008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.935079 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.935083 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.935133 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.939785 (+ 4652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.941877 (+ 2092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.941887 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.941893 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.942129 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.943032 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.943041 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.943047 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.943571 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.943845 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.947098 (+ 3253us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.947470 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Sep 26 20:45:29 dist-test-slave-4d5t krb5kdc[13695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455529, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.253@KRBTEST.COM for kudu/127.23.44.252@KRBTEST.COM
I20260926 20:45:29.957497 13932 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [term 1 LEADER]: Becoming Leader. State: Replica: 48f6b9e010c54bf7baca350eb6133492, State: Running, Role: LEADER
I20260926 20:45:29.958201 13932 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } }
I20260926 20:45:29.967733 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.944983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:36635 (local address 127.0.0.1:39308)
0926 20:45:29.945483 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:29.945509 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:29.945645 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:29.946821 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:29.946830 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:29.947511 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:29.947784 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.947799 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.949753 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.949762 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.951731 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:29.951753 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.953347 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.953356 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:29.953739 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.954533 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:29.954556 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:29.958311 (+ 3755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:29.963830 (+ 5519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.963839 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.963846 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.964617 (+ 771us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.965197 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:29.965205 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:29.965211 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:29.965417 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:29.966293 (+ 876us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:29.966301 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:29.966503 (+ 202us) client_negotiation.cc:769] Sending connection context
0926 20:45:29.966700 (+ 197us) client_negotiation.cc:240] Negotiation successful
0926 20:45:29.967151 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327}
I20260926 20:45:29.969086 13926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:29.945977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39308 (local address 127.23.44.252:36635)
0926 20:45:29.946221 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:29.946230 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:29.946257 (+ 27us) server_negotiation.cc:407] Connection header received
0926 20:45:29.946325 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:29.946333 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:29.946433 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:29.946562 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:29.948002 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.949534 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.951955 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:29.952935 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:29.953084 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:29.958668 (+ 5584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:29.958678 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:29.958686 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:29.958737 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:29.963606 (+ 4869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.964831 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.964839 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.964845 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.965045 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:29.965585 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:29.965593 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:29.965598 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:29.966122 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:29.968087 (+ 1965us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:29.968256 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 20:45:29.968601 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 20:45:29.978018 13849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e: Bootstrap starting.
I20260926 20:45:29.985944 13955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48f6b9e010c54bf7baca350eb6133492. Latest consensus state: current_term: 1 leader_uuid: "48f6b9e010c54bf7baca350eb6133492" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } } }
I20260926 20:45:29.986428 13955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:29.987129 13849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:29.989516 13849 log.cc:824] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e: Log is configured to *not* fsync() on all Append() calls
W20260926 20:45:29.999847 13878 tablet.cc:2369] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:45:30.001397 13961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:30.002785 13849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e: No bootstrap required, opened a new log
I20260926 20:45:30.004011 13961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:30.025108 13969 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 20:45:30.026031 13899 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a [term 1 FOLLOWER]: Refusing update from remote peer 48f6b9e010c54bf7baca350eb6133492: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:30.030011 13955 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:30.030351 13849 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } }
I20260926 20:45:30.030748 13849 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2df151e455b4473a493b878c8174f7e, State: Initialized, Role: FOLLOWER
I20260926 20:45:30.031287 13849 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } }
I20260926 20:45:30.032603 13970 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } } }
I20260926 20:45:30.033273 13970 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:30.033891 13849 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:30.034581 13824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "48f6b9e010c54bf7baca350eb6133492" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c2df151e455b4473a493b878c8174f7e"
I20260926 20:45:30.035036 13824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:30.040434 13824 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48f6b9e010c54bf7baca350eb6133492 in term 1.
I20260926 20:45:30.053742 13824 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e [term 1 FOLLOWER]: Refusing update from remote peer 48f6b9e010c54bf7baca350eb6133492: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:30.037758 13857 master_runner.cc:186] Error getting master registration for 127.23.44.253:35645: OK, instance_id { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" instance_seqno: 1790455529202522 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 20:45:30.057543 13955 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:30.060317 13857 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 20:45:30.062919 13981 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:45:30.063082 13981 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:45:30.075703 13953 sys_catalog.cc:455] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48f6b9e010c54bf7baca350eb6133492. Latest consensus state: current_term: 1 leader_uuid: "48f6b9e010c54bf7baca350eb6133492" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } } }
I20260926 20:45:30.076108 13953 sys_catalog.cc:458] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:30.079281 13967 mvcc.cc:205] Tried to move back new op lower bound from 7333705850974289920 to 7333705850715611136. Current Snapshot: MvccSnapshot[applied={T|T < 7333705850974289920}]
I20260926 20:45:30.090826 13982 mvcc.cc:205] Tried to move back new op lower bound from 7333705850974289920 to 7333705850715611136. Current Snapshot: MvccSnapshot[applied={T|T < 7333705850974289920}]
I20260926 20:45:30.094113 13955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "48f6b9e010c54bf7baca350eb6133492" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } } }
I20260926 20:45:30.094430 13955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:30.094944 13955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "48f6b9e010c54bf7baca350eb6133492" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } } }
I20260926 20:45:30.095316 13955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492 [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:30.096377 13953 sys_catalog.cc:455] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "48f6b9e010c54bf7baca350eb6133492" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } } }
I20260926 20:45:30.096870 13953 sys_catalog.cc:458] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:30.099524 13961 catalog_manager.cc:1347] Generated new cluster ID: ff4f9be759a14b44907d3465bfb84457
I20260926 20:45:30.099665 13961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:30.110190 13988 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "48f6b9e010c54bf7baca350eb6133492" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } } }
I20260926 20:45:30.110766 13988 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:30.111197 13970 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48f6b9e010c54bf7baca350eb6133492. Latest consensus state: current_term: 1 leader_uuid: "48f6b9e010c54bf7baca350eb6133492" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48f6b9e010c54bf7baca350eb6133492" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 40341 } } peers { permanent_uuid: "c2df151e455b4473a493b878c8174f7e" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 35645 } } peers { permanent_uuid: "a279ba4046e349f18c9c73385345a22a" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 36635 } } }
I20260926 20:45:30.111506 13970 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:30.132411 13961 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:30.134830 13961 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:30.171397 13961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48f6b9e010c54bf7baca350eb6133492: Generated new TSK 0
I20260926 20:45:30.172783 13961 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 20:45:30.286540 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:30.273184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52560 (local address 127.23.44.253:35645)
0926 20:45:30.273360 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:30.273365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:30.283541 (+ 10176us) server_negotiation.cc:407] Connection header received
0926 20:45:30.284719 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:30.284727 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:30.284836 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:30.284922 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:30.286276 (+ 1354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52560: BlockingRecv error: recv got EOF from 127.0.0.1:52560 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260926 20:45:30.290604 13926 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:30.274790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39312 (local address 127.23.44.252:36635)
0926 20:45:30.274955 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:30.274960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:30.288676 (+ 13716us) server_negotiation.cc:407] Connection header received
0926 20:45:30.289101 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:30.289107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:30.289200 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:30.289275 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:30.290342 (+ 1067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39312: BlockingRecv error: recv got EOF from 127.0.0.1:39312 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260926 20:45:30.291091 13930 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:30.272812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39116 (local address 127.23.44.254:40341)
0926 20:45:30.273006 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:30.273014 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:30.288386 (+ 15372us) server_negotiation.cc:407] Connection header received
0926 20:45:30.289376 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:30.289392 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:30.289502 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:30.289601 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:30.290830 (+ 1229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39116: BlockingRecv error: recv got EOF from 127.0.0.1:39116 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260926 20:45:30.300625 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:30.298226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52574 (local address 127.23.44.253:35645)
0926 20:45:30.298406 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:30.298413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:30.298922 (+ 509us) server_negotiation.cc:407] Connection header received
0926 20:45:30.299159 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:30.299166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:30.299328 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:30.299460 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:30.300282 (+ 822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52574: BlockingRecv error: recv got EOF from 127.0.0.1:52574 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260926 20:45:30.307912 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:30.305631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52586 (local address 127.23.44.253:35645)
0926 20:45:30.306011 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:30.306018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:30.306205 (+ 187us) server_negotiation.cc:407] Connection header received
0926 20:45:30.306536 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:30.306544 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:30.306646 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:30.306765 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:30.307670 (+ 905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52586: BlockingRecv error: recv got EOF from 127.0.0.1:52586 (error 108)
Metrics: {"server-negotiator.queue_time_us":240}
W20260926 20:45:30.323065 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:30.318962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52588 (local address 127.23.44.253:35645)
0926 20:45:30.319189 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:30.319197 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:30.321289 (+ 2092us) server_negotiation.cc:407] Connection header received
0926 20:45:30.321523 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:30.321529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:30.321663 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:30.321799 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:30.322786 (+ 987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52588: BlockingRecv error: recv got EOF from 127.0.0.1:52588 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260926 20:45:30.335033 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:30.332478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52598 (local address 127.23.44.253:35645)
0926 20:45:30.332787 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:30.332798 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:30.333400 (+ 602us) server_negotiation.cc:407] Connection header received
0926 20:45:30.333519 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:30.333526 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:30.333647 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:30.333780 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:30.334745 (+ 965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52598: BlockingRecv error: recv got EOF from 127.0.0.1:52598 (error 108)
Metrics: {"server-negotiator.queue_time_us":152}
W20260926 20:45:30.357532 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:30.354847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52612 (local address 127.23.44.253:35645)
0926 20:45:30.355096 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:30.355105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:30.355753 (+ 648us) server_negotiation.cc:407] Connection header received
0926 20:45:30.356019 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:30.356027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:30.356143 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:30.356284 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:30.357223 (+ 939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52612: BlockingRecv error: recv got EOF from 127.0.0.1:52612 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260926 20:45:30.381505 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:30.378932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52628 (local address 127.23.44.253:35645)
0926 20:45:30.379163 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:30.379170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:30.379741 (+ 571us) server_negotiation.cc:407] Connection header received
0926 20:45:30.379999 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:30.380005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:30.380108 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:30.380222 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:30.381253 (+ 1031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52628: BlockingRecv error: recv got EOF from 127.0.0.1:52628 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260926 20:45:30.425248 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:30.422717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52642 (local address 127.23.44.253:35645)
0926 20:45:30.422920 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:30.422927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:30.423487 (+ 560us) server_negotiation.cc:407] Connection header received
0926 20:45:30.423775 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:30.423782 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:30.423883 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:30.423996 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:30.424955 (+ 959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52642: BlockingRecv error: recv got EOF from 127.0.0.1:52642 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260926 20:45:30.495838 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:30.493225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52648 (local address 127.23.44.253:35645)
0926 20:45:30.493510 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:30.493517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:30.493906 (+ 389us) server_negotiation.cc:407] Connection header received
0926 20:45:30.494135 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:30.494139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:30.494212 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:30.494299 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:30.495566 (+ 1267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52648: BlockingRecv error: recv got EOF from 127.0.0.1:52648 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260926 20:45:30.629566 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:30.627523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.23.44.253:35645)
0926 20:45:30.627798 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:30.627805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:30.628073 (+ 268us) server_negotiation.cc:407] Connection header received
0926 20:45:30.628318 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:30.628322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:30.628394 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:30.628504 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:30.629274 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52664: BlockingRecv error: recv got EOF from 127.0.0.1:52664 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260926 20:45:30.891642 13855 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:30.889768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.23.44.253:35645)
0926 20:45:30.889986 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:30.889990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:30.890211 (+ 221us) server_negotiation.cc:407] Connection header received
0926 20:45:30.890428 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:30.890433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:30.890515 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:30.890623 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:30.891466 (+ 843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52674: BlockingRecv error: recv got EOF from 127.0.0.1:52674 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 20:45:31.029516 13969 catalog_manager.cc:1259] Loaded cluster ID: ff4f9be759a14b44907d3465bfb84457
I20260926 20:45:31.029639 13969 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a: loading cluster ID for follower catalog manager: success
I20260926 20:45:31.032320 13969 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a: acquiring CA information for follower catalog manager: success
I20260926 20:45:31.034397 13969 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a279ba4046e349f18c9c73385345a22a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 20:45:31.065460 13981 catalog_manager.cc:1259] Loaded cluster ID: ff4f9be759a14b44907d3465bfb84457
I20260926 20:45:31.065557 13981 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e: loading cluster ID for follower catalog manager: success
I20260926 20:45:31.068543 13981 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e: acquiring CA information for follower catalog manager: success
I20260926 20:45:31.071511 13981 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c2df151e455b4473a493b878c8174f7e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 20:45:31.418485 14058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:31.410851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52680 (local address 127.23.44.253:35645)
0926 20:45:31.411510 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:31.411517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:31.411897 (+ 380us) server_negotiation.cc:407] Connection header received
0926 20:45:31.412187 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:31.412191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:31.412370 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:31.412487 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:31.413155 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:31.413885 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:31.416020 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:31.416449 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:31.416578 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:31.417205 (+ 627us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:31.417780 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:31.417892 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:31.418055 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 20:45:31.418160 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":319,"threads_started":1}
I20260926 20:45:31.432164 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:31.425257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39324 (local address 127.23.44.252:36635)
0926 20:45:31.426154 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:31.426159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:31.426173 (+ 14us) server_negotiation.cc:407] Connection header received
0926 20:45:31.426265 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:31.426268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:31.426425 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:31.426605 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:31.427088 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:31.427887 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:31.429692 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:31.430239 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:31.430403 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:31.430926 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:31.431459 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:31.431615 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:31.431716 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 20:45:31.431799 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":535,"threads_started":1}
I20260926 20:45:31.453325 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:31.442654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39332 (local address 127.23.44.252:36635)
0926 20:45:31.443761 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:31.443770 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:31.446800 (+ 3030us) server_negotiation.cc:407] Connection header received
0926 20:45:31.447139 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:31.447144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:31.447237 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:31.447304 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:31.448112 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:31.449040 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:31.450855 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:31.451273 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:31.451407 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:31.452091 (+ 684us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:31.452292 (+ 201us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:31.452405 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:31.452591 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 20:45:31.452691 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":978}
I20260926 20:45:31.453613 14058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:31.441635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52696 (local address 127.23.44.253:35645)
0926 20:45:31.442201 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:31.442210 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:31.446568 (+ 4358us) server_negotiation.cc:407] Connection header received
0926 20:45:31.446845 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:31.446852 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:31.446964 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:31.447096 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:31.447705 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:31.448923 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:31.451521 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:31.452158 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:31.452291 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:31.452848 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:31.452991 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:31.453088 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:31.453205 (+ 117us) server_negotiation.cc:299] Negotiation successful
0926 20:45:31.453287 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415}
I20260926 20:45:31.455093 14072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:31.442706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39118 (local address 127.23.44.254:40341)
0926 20:45:31.447132 (+ 4426us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:31.447147 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:31.447166 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:45:31.447283 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:31.447288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:31.447519 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:31.447697 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:31.448449 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:31.449530 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:31.452161 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:31.452835 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:31.452959 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:31.453696 (+ 737us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:31.454200 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:31.454335 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:31.454447 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 20:45:31.454552 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4305,"thread_start_us":4189,"threads_started":1}
I20260926 20:45:31.467572 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 13708
I20260926 20:45:31.468982 13770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:31.583864 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 13708
W20260926 20:45:31.609766 13875 connection.cc:535] server connection from 127.0.0.1:39248 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 20:45:31.610193 13799 connection.cc:535] server connection from 127.0.0.1:52526 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 20:45:31.610332 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 13782
I20260926 20:45:31.611552 13844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:31.725898 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 13782
I20260926 20:45:31.748641 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 13857
I20260926 20:45:31.750136 13919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:31.864827 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 13857
2026-09-26T20:45:31Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5132 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 20:45:31 dist-test-slave-4d5t krb5kdc[14079](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 20:45:31 dist-test-slave-4d5t krb5kdc[14079](info): set up 2 sockets
Sep 26 20:45:31 dist-test-slave-4d5t krb5kdc[14079](info): commencing operation
krb5kdc: starting...
W20260926 20:45:33.037761 23731 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/test-user.keytab.
Sep 26 20:45:33 dist-test-slave-4d5t krb5kdc[14079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T20:45:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T20:45:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.44.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
Entry for principal kudu/127.23.44.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab.
I20260926 20:45:33.328693 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:44127
--webserver_interface=127.23.44.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:40319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:44127,127.23.44.253:37713,127.23.44.252:38463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 20:45:33.687700 14092 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 20:45:33.688091 14092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:33.688243 14092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:33.688341 14092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:33.696241 14092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:33.696369 14092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:33.696467 14092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:33.696573 14092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:33.696681 14092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:33.708293 14092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:40319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:44127,127.23.44.253:37713,127.23.44.252:38463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.254.keytab
--principal=kudu/127.23.44.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.254:44127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.44.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:33.709877 14092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:33.711391 14092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:33.725077 14097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:33.725013 14100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:33.724994 14098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:33.726332 14092 server_base.cc:1034] running on GCE node
I20260926 20:45:33.727273 14092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:33.729017 14092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:33.730293 14092 hybrid_clock.cc:648] HybridClock initialized: now 1790455533730212 us; error 40 us; skew 500 ppm
I20260926 20:45:33.730731 14092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:33 dist-test-slave-4d5t krb5kdc[14079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455533, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:33.735062 14092 init.cc:377] Logged in from keytab as kudu/127.23.44.254@KRBTEST.COM (short username kudu)
I20260926 20:45:33.737157 14092 webserver.cc:460] Webserver started at http://127.23.44.254:45087/ using document root <none> and password file <none>
I20260926 20:45:33.738190 14092 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:33.738354 14092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:33.738744 14092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:33.742906 14092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/data/instance:
uuid: "c3e4462c48b94d0784d26884c80b132f"
format_stamp: "Formatted at 2026-09-26 20:45:33 on dist-test-slave-4d5t"
server_key: "8e6a5d94a79886b851251fe6f7e5a00b"
server_key_iv: "6064ebd0ab0f4411d4aad10497653c05"
server_key_version: "encryptionkey@0"
I20260926 20:45:33.744226 14092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/wal/instance:
uuid: "c3e4462c48b94d0784d26884c80b132f"
format_stamp: "Formatted at 2026-09-26 20:45:33 on dist-test-slave-4d5t"
server_key: "8e6a5d94a79886b851251fe6f7e5a00b"
server_key_iv: "6064ebd0ab0f4411d4aad10497653c05"
server_key_version: "encryptionkey@0"
I20260926 20:45:33.751260 14092 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260926 20:45:33.757141 14107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:33.759457 14092 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260926 20:45:33.759864 14092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/wal
uuid: "c3e4462c48b94d0784d26884c80b132f"
format_stamp: "Formatted at 2026-09-26 20:45:33 on dist-test-slave-4d5t"
server_key: "8e6a5d94a79886b851251fe6f7e5a00b"
server_key_iv: "6064ebd0ab0f4411d4aad10497653c05"
server_key_version: "encryptionkey@0"
I20260926 20:45:33.760218 14092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:33.788653 14092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:33.793907 14092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:33.794241 14092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:33.819701 14092 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.254:44127
I20260926 20:45:33.819733 14158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.254:44127 every 8 connection(s)
I20260926 20:45:33.821219 14092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/data/info.pb
I20260926 20:45:33.825420 14159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:33.829352 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 14092
I20260926 20:45:33.829584 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-0/wal/instance
I20260926 20:45:33.830042 23731 external_mini_cluster.cc:1428] Setting key a44077be8db2ac927b0f35ccddcf8a21
I20260926 20:45:33.833307 14159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:45:33 dist-test-slave-4d5t krb5kdc[14079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455533, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.44.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.253@KRBTEST.COM" created.
I20260926 20:45:33.861361 14161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:33.836953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47776 (local address 127.23.44.254:44127)
0926 20:45:33.838265 (+ 1312us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:33.838293 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:33.838820 (+ 527us) server_negotiation.cc:407] Connection header received
0926 20:45:33.841717 (+ 2897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:33.841725 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:33.842072 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:33.842326 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:33.844286 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:33.845791 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:33.847416 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:33.848055 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:33.848286 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:33.852046 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:33.852055 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:33.852070 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:33.852106 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:33.855332 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:33.856135 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:33.856142 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:33.856144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:33.856422 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:33.856877 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:33.856881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:33.856883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:33.857865 (+ 982us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:33.858232 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:33.859112 (+ 880us) server_negotiation.cc:299] Negotiation successful
0926 20:45:33.859592 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":1047,"thread_start_us":710,"threads_started":1}
I20260926 20:45:33.861387 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:33.836957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44127 (local address 127.0.0.1:47776)
0926 20:45:33.838628 (+ 1671us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:33.838681 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:33.841122 (+ 2441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:33.842348 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:33.842356 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:33.843503 (+ 1147us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:33.843939 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:33.843966 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:33.845990 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:33.846006 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:33.847244 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:33.847258 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:33.848323 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:33.848330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:33.848519 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:33.849324 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:33.849349 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:33.851777 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:33.855552 (+ 3775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:33.855563 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:33.855576 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:33.855998 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:33.856527 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:33.856530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:33.856532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:33.856747 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:33.858134 (+ 1387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:33.858144 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:33.858569 (+ 425us) client_negotiation.cc:769] Sending connection context
0926 20:45:33.858935 (+ 366us) client_negotiation.cc:240] Negotiation successful
0926 20:45:33.859740 (+ 805us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1458,"mutex_wait_us":225,"spinlock_wait_cycles":20864,"thread_start_us":1221,"threads_started":1}
I20260926 20:45:33.865315 14159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:45:33.866830 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:33.866223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:37713 (local address 127.0.0.1:36336)
0926 20:45:33.866464 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:33.866621 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260926 20:45:33.867280 14111 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
I20260926 20:45:33.869107 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:33.868734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:37713 (local address 127.0.0.1:36352)
0926 20:45:33.868888 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:33.868963 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 20:45:33.869361 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:37713: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.253.keytab.
Entry for principal kudu/127.23.44.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.253.keytab.
I20260926 20:45:33.893234 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.253:37713
--webserver_interface=127.23.44.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.253.keytab
--principal=kudu/127.23.44.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:40319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:44127,127.23.44.253:37713,127.23.44.252:38463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:45:33.918743 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } attempt: 1
I20260926 20:45:33.919950 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:33.919493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:37713 (local address 127.0.0.1:36354)
0926 20:45:33.919700 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:33.919824 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260926 20:45:33.921104 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:33.920748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:37713 (local address 127.0.0.1:36360)
0926 20:45:33.920895 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:33.920965 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260926 20:45:33.921487 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:37713: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
I20260926 20:45:33.989861 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } attempt: 2
I20260926 20:45:33.991071 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:33.990566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:37713 (local address 127.0.0.1:36370)
0926 20:45:33.990827 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:33.990914 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 20:45:33.992211 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:33.991792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:37713 (local address 127.0.0.1:36386)
0926 20:45:33.991985 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:33.992068 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 20:45:33.992404 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:37713: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
I20260926 20:45:34.083727 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } attempt: 3
I20260926 20:45:34.085119 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.084543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:37713 (local address 127.0.0.1:36392)
0926 20:45:34.084842 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.084972 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 20:45:34.086251 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.085845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:37713 (local address 127.0.0.1:36400)
0926 20:45:34.086029 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.086102 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260926 20:45:34.086478 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:37713: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
I20260926 20:45:34.229769 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } attempt: 4
I20260926 20:45:34.231017 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.230452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:37713 (local address 127.0.0.1:36414)
0926 20:45:34.230764 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.230862 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260926 20:45:34.232034 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.231668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:37713 (local address 127.0.0.1:36424)
0926 20:45:34.231830 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.231914 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 20:45:34.232230 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.253:37713: Network error: Client connection negotiation failed: client connection to 127.23.44.253:37713: connect: Connection refused (error 111)
W20260926 20:45:34.260748 14166 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 20:45:34.261147 14166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:34.261272 14166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:34.261413 14166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:34.269104 14166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:34.269232 14166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:34.269354 14166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:34.269448 14166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:34.269546 14166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:34.280515 14166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:40319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:44127,127.23.44.253:37713,127.23.44.252:38463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.253.keytab
--principal=kudu/127.23.44.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.253:37713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.44.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:34.281940 14166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:34.283274 14166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:34.296630 14171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:34.296545 14172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:34.296799 14174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:34.298301 14166 server_base.cc:1034] running on GCE node
I20260926 20:45:34.299182 14166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:34.300807 14166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:34.302038 14166 hybrid_clock.cc:648] HybridClock initialized: now 1790455534301965 us; error 46 us; skew 500 ppm
I20260926 20:45:34.302419 14166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:34 dist-test-slave-4d5t krb5kdc[14079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455534, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:34.306069 14166 init.cc:377] Logged in from keytab as kudu/127.23.44.253@KRBTEST.COM (short username kudu)
I20260926 20:45:34.307849 14166 webserver.cc:460] Webserver started at http://127.23.44.253:39803/ using document root <none> and password file <none>
I20260926 20:45:34.308722 14166 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:34.308871 14166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:34.309222 14166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:34.312713 14166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/data/instance:
uuid: "362b603c03754702aa46a82ff2322aa6"
format_stamp: "Formatted at 2026-09-26 20:45:34 on dist-test-slave-4d5t"
server_key: "9d8b3b712a6440c0586c74303c320923"
server_key_iv: "72b87ac45c435b6e8ee2385c45b3d5b6"
server_key_version: "encryptionkey@0"
I20260926 20:45:34.313627 14166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/wal/instance:
uuid: "362b603c03754702aa46a82ff2322aa6"
format_stamp: "Formatted at 2026-09-26 20:45:34 on dist-test-slave-4d5t"
server_key: "9d8b3b712a6440c0586c74303c320923"
server_key_iv: "72b87ac45c435b6e8ee2385c45b3d5b6"
server_key_version: "encryptionkey@0"
I20260926 20:45:34.319290 14166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 20:45:34.323417 14181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:34.325068 14166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 20:45:34.325276 14166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/wal
uuid: "362b603c03754702aa46a82ff2322aa6"
format_stamp: "Formatted at 2026-09-26 20:45:34 on dist-test-slave-4d5t"
server_key: "9d8b3b712a6440c0586c74303c320923"
server_key_iv: "72b87ac45c435b6e8ee2385c45b3d5b6"
server_key_version: "encryptionkey@0"
I20260926 20:45:34.325517 14166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:34.341991 14166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:34.346527 14166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:34.346832 14166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:34.366876 14166 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.253:37713
I20260926 20:45:34.366899 14232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.253:37713 every 8 connection(s)
I20260926 20:45:34.368314 14166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/data/info.pb
I20260926 20:45:34.371690 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 14166
I20260926 20:45:34.371879 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-1/wal/instance
I20260926 20:45:34.372423 23731 external_mini_cluster.cc:1428] Setting key b7a1115b004e6aea72465e1a16182309
I20260926 20:45:34.372599 14233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:34.383057 14233 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:45:34 dist-test-slave-4d5t krb5kdc[14079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455534, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.253@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.44.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.44.252@KRBTEST.COM" created.
I20260926 20:45:34.407743 14236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.386418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47786 (local address 127.23.44.254:44127)
0926 20:45:34.386975 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:34.386983 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:34.388037 (+ 1054us) server_negotiation.cc:407] Connection header received
0926 20:45:34.390162 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:34.390172 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:34.390275 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:34.390415 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:34.392080 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.393337 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.395014 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.395792 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.395949 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.399881 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:34.399889 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:34.399895 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:34.399962 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:34.403439 (+ 3477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.404522 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.404534 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.404536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.404796 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.405384 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.405389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.405391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.405709 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:34.405849 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:34.406980 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0926 20:45:34.407242 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":265,"threads_started":1}
I20260926 20:45:34.409284 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.387000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44127 (local address 127.0.0.1:47786)
0926 20:45:34.387836 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.387884 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:34.389842 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:34.390621 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:34.390630 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:34.391212 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:34.391819 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.391837 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.393533 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.393537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:34.394815 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.394840 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.396051 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.396056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:34.396319 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.397183 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:34.397219 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:34.399567 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:34.403667 (+ 4100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:34.403676 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:34.403693 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:34.404352 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:34.404924 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:34.404930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:34.404934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:34.405186 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:34.405888 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:34.405901 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:34.406460 (+ 559us) client_negotiation.cc:769] Sending connection context
0926 20:45:34.406832 (+ 372us) client_negotiation.cc:240] Negotiation successful
0926 20:45:34.407462 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":358,"threads_started":1}
I20260926 20:45:34.411255 14233 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:45:34 dist-test-slave-4d5t krb5kdc[14079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455534, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.253@KRBTEST.COM for kudu/127.23.44.253@KRBTEST.COM
I20260926 20:45:34.429929 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.412282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:37713 (local address 127.0.0.1:36434)
0926 20:45:34.412489 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.412506 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:34.412701 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:34.413614 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:34.413619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:34.414099 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:34.414229 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.414237 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.415659 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.415667 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:34.417159 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.417172 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.418492 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.418498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:34.418678 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.419352 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:34.419370 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:34.421684 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:34.426511 (+ 4827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:34.426517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:34.426521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:34.427102 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:34.427775 (+ 673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:34.427780 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:34.427783 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:34.427988 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:34.428887 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:34.428891 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:34.429058 (+ 167us) client_negotiation.cc:769] Sending connection context
0926 20:45:34.429245 (+ 187us) client_negotiation.cc:240] Negotiation successful
0926 20:45:34.429550 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260926 20:45:34.430428 14238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.412310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36434 (local address 127.23.44.253:37713)
0926 20:45:34.413175 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:34.413181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:34.413207 (+ 26us) server_negotiation.cc:407] Connection header received
0926 20:45:34.413253 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:34.413258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:34.413445 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:34.413616 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:34.414415 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.415511 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.417304 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.418298 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.418448 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.421959 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:34.421981 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:34.422005 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:34.422051 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:34.426217 (+ 4166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.427350 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.427363 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.427369 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.427581 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.428144 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.428151 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.428157 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.428693 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:34.428956 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:34.429434 (+ 478us) server_negotiation.cc:299] Negotiation successful
0926 20:45:34.429752 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":536,"threads_started":1}
I20260926 20:45:34.433152 14233 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:45:34.434571 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.434020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59276)
0926 20:45:34.434222 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.434372 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260926 20:45:34.434944 14182 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
I20260926 20:45:34.436826 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.436358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59292)
0926 20:45:34.436540 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.436709 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260926 20:45:34.437105 14233 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:38463: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.44.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.252.keytab.
Entry for principal kudu/127.23.44.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.252.keytab.
I20260926 20:45:34.444876 23731 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
/tmp/dist-test-taskO4QBsv/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.252:38463
--webserver_interface=127.23.44.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.252.keytab
--principal=kudu/127.23.44.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.44.212:40319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.44.254:44127,127.23.44.253:37713,127.23.44.252:38463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 20:45:34.486515 14233 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } attempt: 1
I20260926 20:45:34.487845 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.487269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59306)
0926 20:45:34.487550 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.487673 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260926 20:45:34.489153 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.488682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59320)
0926 20:45:34.488935 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.489027 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260926 20:45:34.489390 14233 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:38463: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
I20260926 20:45:34.531522 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } attempt: 5
Sep 26 20:45:34 dist-test-slave-4d5t krb5kdc[14079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455533, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for kudu/127.23.44.253@KRBTEST.COM
I20260926 20:45:34.548906 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.532293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:37713 (local address 127.0.0.1:36442)
0926 20:45:34.532629 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.532652 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:34.532784 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:34.533327 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:34.533332 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:34.533852 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:34.534093 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.534105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.535255 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.535260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:34.536638 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.536653 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.537519 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.537525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:34.537753 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.538501 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:34.538527 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:34.541346 (+ 2819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:34.545569 (+ 4223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:34.545575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:34.545578 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:34.546047 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:34.546819 (+ 772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:34.546827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:34.546834 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:34.547101 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:34.547837 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:34.547841 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:34.548031 (+ 190us) client_negotiation.cc:769] Sending connection context
0926 20:45:34.548170 (+ 139us) client_negotiation.cc:240] Negotiation successful
0926 20:45:34.548490 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168}
I20260926 20:45:34.548944 14238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.532676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36442 (local address 127.23.44.253:37713)
0926 20:45:34.532895 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:34.532900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:34.532924 (+ 24us) server_negotiation.cc:407] Connection header received
0926 20:45:34.533003 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:34.533008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:34.533108 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:34.533241 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:34.534331 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.535101 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.536804 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.537353 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.537463 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.541651 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:34.541660 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:34.541667 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:34.541744 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:34.545306 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.546325 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.546333 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.546413 (+ 80us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.546604 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.547270 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.547275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.547278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.547668 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:34.547839 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:34.548325 (+ 486us) server_negotiation.cc:299] Negotiation successful
0926 20:45:34.548604 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:45:34.550282 14159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:45:34.551533 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.551068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59326)
0926 20:45:34.551268 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.551366 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 20:45:34.552724 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.552277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59342)
0926 20:45:34.552478 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.552621 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260926 20:45:34.553025 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:38463: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
I20260926 20:45:34.557636 14233 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } attempt: 2
I20260926 20:45:34.558724 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.558216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59348)
0926 20:45:34.558430 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.558531 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 20:45:34.559746 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.559349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59362)
0926 20:45:34.559508 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.559594 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 20:45:34.560168 14233 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:38463: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
I20260926 20:45:34.604343 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } attempt: 1
I20260926 20:45:34.606076 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.605441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59376)
0926 20:45:34.605742 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.605852 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260926 20:45:34.607409 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.606993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59390)
0926 20:45:34.607210 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.607284 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260926 20:45:34.607647 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:38463: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
I20260926 20:45:34.651491 14233 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } attempt: 3
I20260926 20:45:34.652861 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.652324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59394)
0926 20:45:34.652599 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.652700 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 20:45:34.653972 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.653548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59406)
0926 20:45:34.653771 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.653857 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260926 20:45:34.654165 14233 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:38463: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
I20260926 20:45:34.675909 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } attempt: 2
I20260926 20:45:34.677294 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.676765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59410)
0926 20:45:34.677014 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.677103 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260926 20:45:34.679077 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.678482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59416)
0926 20:45:34.678715 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.678932 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260926 20:45:34.679376 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:38463: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
I20260926 20:45:34.785682 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } attempt: 3
I20260926 20:45:34.787258 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.786523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59420)
0926 20:45:34.786879 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.787023 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260926 20:45:34.788880 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.788271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59434)
0926 20:45:34.788540 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.788717 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260926 20:45:34.789144 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:38463: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
I20260926 20:45:34.797402 14233 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } attempt: 4
I20260926 20:45:34.798736 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.798075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59446)
0926 20:45:34.798370 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.798505 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 20:45:34.800020 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.799550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59454)
0926 20:45:34.799743 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.799854 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260926 20:45:34.800304 14233 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.44.252:38463: Network error: Client connection negotiation failed: client connection to 127.23.44.252:38463: connect: Connection refused (error 111)
W20260926 20:45:34.816185 14241 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 20:45:34.816609 14241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 20:45:34.816738 14241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 20:45:34.816840 14241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 20:45:34.828178 14241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 20:45:34.828310 14241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 20:45:34.828388 14241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 20:45:34.828449 14241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 20:45:34.828536 14241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 20:45:34.839960 14241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.44.212:40319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.44.254:44127,127.23.44.253:37713,127.23.44.252:38463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/krb5kdc/kudu_127.23.44.252.keytab
--principal=kudu/127.23.44.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.44.252:38463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.44.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 19:19:52 UTC on ccycloud.quasar-vsunte.root.comops.site
build id 4675017
ASAN enabled
I20260926 20:45:34.841356 14241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 20:45:34.842686 14241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 20:45:34.855357 14249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:34.855450 14247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 20:45:34.855386 14246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 20:45:34.856652 14241 server_base.cc:1034] running on GCE node
I20260926 20:45:34.857561 14241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 20:45:34.859140 14241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 20:45:34.860522 14241 hybrid_clock.cc:648] HybridClock initialized: now 1790455534860409 us; error 31 us; skew 500 ppm
I20260926 20:45:34.861129 14241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 20:45:34 dist-test-slave-4d5t krb5kdc[14079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455534, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 20:45:34.865088 14241 init.cc:377] Logged in from keytab as kudu/127.23.44.252@KRBTEST.COM (short username kudu)
I20260926 20:45:34.866729 14241 webserver.cc:460] Webserver started at http://127.23.44.252:39595/ using document root <none> and password file <none>
I20260926 20:45:34.867419 14241 fs_manager.cc:359] Metadata directory not provided
I20260926 20:45:34.867641 14241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 20:45:34.867961 14241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 20:45:34.871094 14241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/data/instance:
uuid: "1a78bd4b1b1b47b9ba566664d92508f4"
format_stamp: "Formatted at 2026-09-26 20:45:34 on dist-test-slave-4d5t"
server_key: "e9032a51c0274f0a7233ec64fa9c49ae"
server_key_iv: "3aa3b2b4da26513efd3cc39cb2eb9aa4"
server_key_version: "encryptionkey@0"
I20260926 20:45:34.871879 14241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/wal/instance:
uuid: "1a78bd4b1b1b47b9ba566664d92508f4"
format_stamp: "Formatted at 2026-09-26 20:45:34 on dist-test-slave-4d5t"
server_key: "e9032a51c0274f0a7233ec64fa9c49ae"
server_key_iv: "3aa3b2b4da26513efd3cc39cb2eb9aa4"
server_key_version: "encryptionkey@0"
I20260926 20:45:34.877328 14241 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260926 20:45:34.881191 14256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 20:45:34.882802 14241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260926 20:45:34.882961 14241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/data,/tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/wal
uuid: "1a78bd4b1b1b47b9ba566664d92508f4"
format_stamp: "Formatted at 2026-09-26 20:45:34 on dist-test-slave-4d5t"
server_key: "e9032a51c0274f0a7233ec64fa9c49ae"
server_key_iv: "3aa3b2b4da26513efd3cc39cb2eb9aa4"
server_key_version: "encryptionkey@0"
I20260926 20:45:34.883167 14241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 20:45:34.905093 14241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 20:45:34.909843 14241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 20:45:34.910081 14241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 20:45:34.930595 14241 rpc_server.cc:307] RPC server started. Bound to: 127.23.44.252:38463
I20260926 20:45:34.930626 14307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.44.252:38463 every 8 connection(s)
I20260926 20:45:34.932029 14241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/data/info.pb
I20260926 20:45:34.935425 23731 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu as pid 14241
I20260926 20:45:34.935616 23731 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO4QBsv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790455151607908-23731-0/minicluster-data/master-2/wal/instance
I20260926 20:45:34.936136 14308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 20:45:34.936156 23731 external_mini_cluster.cc:1428] Setting key c329007bea0d65205819c64ed0b66384
I20260926 20:45:34.945779 14308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:45:34 dist-test-slave-4d5t krb5kdc[14079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.252@KRBTEST.COM
Sep 26 20:45:34 dist-test-slave-4d5t krb5kdc[14079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455534, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.252@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
I20260926 20:45:34.963039 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.938427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59466 (local address 127.23.44.252:38463)
0926 20:45:34.939325 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:34.939337 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:34.939384 (+ 47us) server_negotiation.cc:407] Connection header received
0926 20:45:34.941553 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:34.941578 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:34.942049 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:34.942516 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:34.944175 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.945846 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.947302 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.948299 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.948521 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.953527 (+ 5006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:34.953538 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:34.953553 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:34.953599 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:34.957717 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.958842 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.958854 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.958868 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.959118 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.959617 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.959621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.959645 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.960520 (+ 875us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:34.960884 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:34.961363 (+ 479us) server_negotiation.cc:299] Negotiation successful
0926 20:45:34.961776 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":416,"threads_started":1}
I20260926 20:45:34.966369 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } attempt: 4
I20260926 20:45:34.970052 14313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.948436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.254:44127 (local address 127.0.0.1:47798)
0926 20:45:34.952038 (+ 3602us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.952086 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:34.952329 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:34.952863 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:34.952871 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:34.953749 (+ 878us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:34.954199 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.954218 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.955652 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.955661 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:34.957528 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.957546 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.958921 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.958930 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:34.959144 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.959874 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:34.959898 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:34.963014 (+ 3116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:34.966782 (+ 3768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:34.966793 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:34.966799 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:34.967419 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:34.967929 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:34.967936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:34.967941 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:34.968128 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:34.968862 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:34.968870 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:34.969069 (+ 199us) client_negotiation.cc:769] Sending connection context
0926 20:45:34.969259 (+ 190us) client_negotiation.cc:240] Negotiation successful
0926 20:45:34.969591 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3368,"thread_start_us":3239,"threads_started":1}
I20260926 20:45:34.970041 14312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.948399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47798 (local address 127.23.44.254:44127)
0926 20:45:34.949719 (+ 1320us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:34.949726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:34.952195 (+ 2469us) server_negotiation.cc:407] Connection header received
0926 20:45:34.952478 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:34.952487 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:34.952646 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:34.952773 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:34.954445 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.955414 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.957729 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.958556 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.958742 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.963244 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:34.963251 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:34.963255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:34.963302 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:34.966610 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.967568 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.967574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.967577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.967760 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.968256 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.968263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.968267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.968730 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:34.968888 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:34.969369 (+ 481us) server_negotiation.cc:299] Negotiation successful
0926 20:45:34.969578 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":1170,"thread_start_us":983,"threads_started":1}
I20260926 20:45:34.971541 14308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 20:45:34 dist-test-slave-4d5t krb5kdc[14079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455533, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.254@KRBTEST.COM for kudu/127.23.44.252@KRBTEST.COM
Sep 26 20:45:34 dist-test-slave-4d5t krb5kdc[14079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.254@KRBTEST.COM
Sep 26 20:45:34 dist-test-slave-4d5t krb5kdc[14079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455534, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.252@KRBTEST.COM for kudu/127.23.44.253@KRBTEST.COM
I20260926 20:45:34.984407 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.967431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59472 (local address 127.23.44.252:38463)
0926 20:45:34.967634 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:34.967640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:34.967657 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:45:34.967706 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:34.967711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:34.967776 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:34.967891 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:34.968940 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.970069 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.972114 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.972839 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.973015 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.977441 (+ 4426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:34.977446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:34.977449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:34.977478 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:34.981215 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.982061 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.982065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.982067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.982275 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.982749 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.982755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.982759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.983181 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:34.983375 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:34.983811 (+ 436us) server_negotiation.cc:299] Negotiation successful
0926 20:45:34.984039 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 20:45:34.984454 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.966899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59472)
0926 20:45:34.967235 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.967256 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:34.967356 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:34.967918 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:34.967923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:34.968439 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:34.968763 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.968777 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.970604 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.970609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:34.971943 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.971977 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.973028 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.973033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:34.973359 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.974068 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:34.974085 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:34.977178 (+ 3093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:34.981401 (+ 4223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:34.981407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:34.981411 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:34.981913 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:34.982409 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:34.982415 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:34.982419 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:34.982615 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:34.983386 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:34.983392 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:34.983564 (+ 172us) client_negotiation.cc:769] Sending connection context
0926 20:45:34.983721 (+ 157us) client_negotiation.cc:240] Negotiation successful
0926 20:45:34.984039 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":23}
I20260926 20:45:34.986159 14314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.966539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47808 (local address 127.23.44.254:44127)
0926 20:45:34.967959 (+ 1420us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:34.967964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:34.967987 (+ 23us) server_negotiation.cc:407] Connection header received
0926 20:45:34.968031 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:34.968035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:34.968104 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:34.968213 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:34.969844 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.971108 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.972849 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.973594 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.973752 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.978418 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:34.978425 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:34.978429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:34.978471 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:34.982758 (+ 4287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.983682 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.983690 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.983696 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.983877 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.984377 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.984383 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.984386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.984787 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:34.985077 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:34.985554 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 20:45:34.985769 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1267,"thread_start_us":1041,"threads_started":1}
I20260926 20:45:34.987964 14313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.972370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.253:37713 (local address 127.0.0.1:36452)
0926 20:45:34.972543 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.972581 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:34.972680 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:34.973469 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:34.973475 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:34.973951 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:34.974146 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.974161 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.975729 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.975737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:34.976869 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.976888 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.978103 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.978108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:34.978309 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.978789 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:34.978805 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:34.981252 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:34.984960 (+ 3708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:34.984966 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:34.984978 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:34.985550 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:34.986043 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:34.986047 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:34.986050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:34.986237 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:34.987013 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:34.987021 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:34.987190 (+ 169us) client_negotiation.cc:769] Sending connection context
0926 20:45:34.987332 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 20:45:34.987615 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 20:45:34.988586 14238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.972832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36452 (local address 127.23.44.253:37713)
0926 20:45:34.973007 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:34.973013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:34.973030 (+ 17us) server_negotiation.cc:407] Connection header received
0926 20:45:34.973101 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:34.973108 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:34.973195 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:34.973296 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:34.974547 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.975579 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.977057 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.977741 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.977843 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.981461 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:34.981466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:34.981469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:34.981502 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:34.984809 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.985713 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.985719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.985722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.985889 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:34.986368 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:34.986372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:34.986375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:34.986748 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:34.986900 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:34.987640 (+ 740us) server_negotiation.cc:299] Negotiation successful
0926 20:45:34.987951 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 20:45:34.990937 14159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f: Bootstrap starting.
I20260926 20:45:34.994132 14308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } has no permanent_uuid. Determining permanent_uuid...
I20260926 20:45:34.995640 14159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:34.997104 14159 log.cc:824] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f: Log is configured to *not* fsync() on all Append() calls
Sep 26 20:45:34 dist-test-slave-4d5t krb5kdc[14079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.44.253@KRBTEST.COM
I20260926 20:45:35.000473 14159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f: No bootstrap required, opened a new log
Sep 26 20:45:35 dist-test-slave-4d5t krb5kdc[14079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455534, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.252@KRBTEST.COM for kudu/127.23.44.252@KRBTEST.COM
I20260926 20:45:35.008265 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.988121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36466 (local address 127.23.44.253:37713)
0926 20:45:34.988747 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:34.988754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:34.988898 (+ 144us) server_negotiation.cc:407] Connection header received
0926 20:45:34.989084 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:34.989090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:34.989164 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:34.989279 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:34.990562 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.991466 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.992761 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.993438 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.993593 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:34.999689 (+ 6096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:34.999697 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:34.999703 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:34.999785 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:35.004320 (+ 4535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:35.005246 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:35.005252 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:35.005256 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:35.005578 (+ 322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:35.006193 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:35.006199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:35.006203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:35.006713 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:35.007216 (+ 503us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:35.007533 (+ 317us) server_negotiation.cc:299] Negotiation successful
0926 20:45:35.007802 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":279,"threads_started":1}
I20260926 20:45:35.011684 14159 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } }
I20260926 20:45:35.012344 14159 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3e4462c48b94d0784d26884c80b132f, State: Initialized, Role: FOLLOWER
I20260926 20:45:35.013110 14159 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } }
I20260926 20:45:35.014417 23731 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 20:45:35.014609 14313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.995114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59480)
0926 20:45:34.995319 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:34.995339 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:34.995453 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:34.995958 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:34.995964 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:34.996455 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:34.996681 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.996695 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.998156 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.998164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:34.999199 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:34.999211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.001162 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.001169 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:35.001462 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:35.002423 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:35.002456 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:35.005434 (+ 2978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:35.010924 (+ 5490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:35.010930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:35.010935 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:35.011821 (+ 886us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:35.012257 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:35.012263 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:35.012267 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:35.012401 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:35.013098 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:35.013108 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:35.013456 (+ 348us) client_negotiation.cc:769] Sending connection context
0926 20:45:35.013674 (+ 218us) client_negotiation.cc:240] Negotiation successful
0926 20:45:35.014165 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260926 20:45:35.014600 14317 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } } }
I20260926 20:45:35.015358 14317 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:35.015841 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:34.995400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59480 (local address 127.23.44.252:38463)
0926 20:45:34.995594 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:34.995602 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:34.995621 (+ 19us) server_negotiation.cc:407] Connection header received
0926 20:45:34.995673 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:34.995680 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:34.995763 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:34.995910 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:34.996844 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:34.997987 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:34.999647 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.000488 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.002765 (+ 2277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:35.005740 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:35.005750 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:35.005758 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:35.005806 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:35.010744 (+ 4938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:35.011973 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:35.011978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:35.011982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:35.012156 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:35.012509 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:35.012517 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:35.012520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:35.012966 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:35.013117 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:35.015188 (+ 2071us) server_negotiation.cc:299] Negotiation successful
0926 20:45:35.015486 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 20:45:35.016631 14159 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:35.017285 14134 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c3e4462c48b94d0784d26884c80b132f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47808
I20260926 20:45:35.017729 14134 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 20:45:35.017921 14134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:35.021190 14308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4: Bootstrap starting.
I20260926 20:45:35.023061 14134 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } }
I20260926 20:45:35.024415 14308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:35.025653 14134 leader_election.cc:288] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [CANDIDATE]: Term 1 election: Requested vote from peers 362b603c03754702aa46a82ff2322aa6 (127.23.44.253:37713), 1a78bd4b1b1b47b9ba566664d92508f4 (127.23.44.252:38463)
I20260926 20:45:35.025878 14308 log.cc:824] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:35.031878 14308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4: No bootstrap required, opened a new log
W20260926 20:45:35.047470 14113 tablet.cc:2369] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:45:35.047935 14308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } }
I20260926 20:45:35.048943 14308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a78bd4b1b1b47b9ba566664d92508f4, State: Initialized, Role: FOLLOWER
I20260926 20:45:35.050030 14308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } }
I20260926 20:45:35.077816 14338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } } }
I20260926 20:45:35.078372 14338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:35.079882 14308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:35.079936 14283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c3e4462c48b94d0784d26884c80b132f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1a78bd4b1b1b47b9ba566664d92508f4"
I20260926 20:45:35.080521 14283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:35.084440 14283 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3e4462c48b94d0784d26884c80b132f in term 1.
I20260926 20:45:35.085392 14108 leader_election.cc:302] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a78bd4b1b1b47b9ba566664d92508f4, c3e4462c48b94d0784d26884c80b132f; no voters:
I20260926 20:45:35.086023 14317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 20:45:35.094280 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:35.036184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36472 (local address 127.23.44.253:37713)
0926 20:45:35.036514 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.036525 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.071934 (+ 35409us) server_negotiation.cc:407] Connection header received
0926 20:45:35.072249 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.072257 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.072518 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.072678 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:35.075508 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.076670 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.080126 (+ 3456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.080832 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.080953 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:35.084281 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:35.084290 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:35.084298 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:35.084346 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:35.088651 (+ 4305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:35.090299 (+ 1648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:35.090309 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:35.090315 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:35.090510 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:35.091065 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:35.091069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:35.091090 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:35.091601 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:35.092122 (+ 521us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:35.093223 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0926 20:45:35.093673 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 20:45:35.038844 14092 master_runner.cc:186] Error getting master registration for 127.23.44.253:37713: OK, instance_id { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" instance_seqno: 1790455534361160 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 20:45:35.099658 14092 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:45:35.099526 14233 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } attempt: 5
W20260926 20:45:35.105116 14333 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:45:35.105434 14333 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 20:45:35.120182 14317 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [term 1 LEADER]: Becoming Leader. State: Replica: c3e4462c48b94d0784d26884c80b132f, State: Running, Role: LEADER
I20260926 20:45:35.120958 14317 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } }
W20260926 20:45:35.124534 14351 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 20:45:35.124693 14351 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 26 20:45:35 dist-test-slave-4d5t krb5kdc[14079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790455534, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.44.253@KRBTEST.COM for kudu/127.23.44.252@KRBTEST.COM
I20260926 20:45:35.138749 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:35.101028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59488 (local address 127.23.44.252:38463)
0926 20:45:35.114133 (+ 13105us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.114141 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.114179 (+ 38us) server_negotiation.cc:407] Connection header received
0926 20:45:35.114265 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.114274 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.114635 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.114908 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:35.116244 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.117741 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.120176 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.121104 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.121209 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:35.128332 (+ 7123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:35.128339 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:35.128347 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:35.128388 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:35.132909 (+ 4521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:35.134279 (+ 1370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:35.134285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:35.134290 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:35.134472 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:35.135118 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:35.135125 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:35.135130 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:35.135842 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:35.136070 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:35.137510 (+ 1440us) server_negotiation.cc:299] Negotiation successful
0926 20:45:35.137944 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12870,"spinlock_wait_cycles":23296,"thread_start_us":14231,"threads_started":1}
I20260926 20:45:35.139843 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:35.043925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59484 (local address 127.23.44.252:38463)
0926 20:45:35.065311 (+ 21386us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.065319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.074365 (+ 9046us) server_negotiation.cc:407] Connection header received
0926 20:45:35.075828 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.075834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.075928 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.076004 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:35.107434 (+ 31430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.109004 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.125492 (+ 16488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.126260 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.126398 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:35.132544 (+ 6146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:35.132580 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:35.132590 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:35.132636 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:35.136458 (+ 3822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:35.137363 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:35.137368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:35.137371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:35.137534 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:35.138013 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:35.138017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:35.138021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:35.138399 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:35.138644 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:35.139262 (+ 618us) server_negotiation.cc:299] Negotiation successful
0926 20:45:35.139505 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21225}
I20260926 20:45:35.140268 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:35.100217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.44.252:38463 (local address 127.0.0.1:59488)
0926 20:45:35.100654 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0926 20:45:35.100675 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 20:45:35.100789 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 20:45:35.115005 (+ 14216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 20:45:35.115015 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 20:45:35.115725 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0926 20:45:35.115984 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:35.116003 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.118121 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.118129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:35.119891 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 20:45:35.119911 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.121974 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.121980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 20:45:35.122252 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:35.123090 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 20:45:35.123115 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 20:45:35.127995 (+ 4880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 20:45:35.133193 (+ 5198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:35.133199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:35.133224 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:35.134068 (+ 844us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:35.134661 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 20:45:35.134667 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 20:45:35.134671 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 20:45:35.134926 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 20:45:35.136336 (+ 1410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 20:45:35.136344 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 20:45:35.136711 (+ 367us) client_negotiation.cc:769] Sending connection context
0926 20:45:35.139157 (+ 2446us) client_negotiation.cc:240] Negotiation successful
0926 20:45:35.139673 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258}
I20260926 20:45:35.145016 14340 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3e4462c48b94d0784d26884c80b132f. Latest consensus state: current_term: 1 leader_uuid: "c3e4462c48b94d0784d26884c80b132f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } } }
I20260926 20:45:35.145645 14340 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:35.146368 14233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6: Bootstrap starting.
I20260926 20:45:35.150780 14354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 20:45:35.153208 14354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 20:45:35.140937 14241 master_runner.cc:186] Error getting master registration for 127.23.44.253:37713: OK, instance_id { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" instance_seqno: 1790455534361160 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 20:45:35.154078 14241 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 20:45:35.155705 14233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6: Neither blocks nor log segments found. Creating new log.
I20260926 20:45:35.157335 14233 log.cc:824] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6: Log is configured to *not* fsync() on all Append() calls
I20260926 20:45:35.160472 14314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:35.032668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47818 (local address 127.23.44.254:44127)
0926 20:45:35.033144 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.033154 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.064266 (+ 31112us) server_negotiation.cc:407] Connection header received
0926 20:45:35.066024 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.066034 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.066169 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.066261 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:35.103175 (+ 36914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.104625 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.123734 (+ 19109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.124412 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.124530 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:35.130202 (+ 5672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 20:45:35.130210 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 20:45:35.130218 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 20:45:35.130259 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 20:45:35.134223 (+ 3964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:35.149674 (+ 15451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:35.149685 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:35.149691 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:35.149926 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 20:45:35.158450 (+ 8524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 20:45:35.158457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 20:45:35.158461 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 20:45:35.158914 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 20:45:35.159190 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:35.159801 (+ 611us) server_negotiation.cc:299] Negotiation successful
0926 20:45:35.160089 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
W20260926 20:45:35.162865 14262 tablet.cc:2369] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 20:45:35.167675 14233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6: No bootstrap required, opened a new log
I20260926 20:45:35.179584 14233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } }
I20260926 20:45:35.180164 14233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 362b603c03754702aa46a82ff2322aa6, State: Initialized, Role: FOLLOWER
I20260926 20:45:35.180835 14233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } }
I20260926 20:45:35.192586 14233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 20:45:35.192749 14208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c3e4462c48b94d0784d26884c80b132f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "362b603c03754702aa46a82ff2322aa6"
I20260926 20:45:35.193454 14208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 20:45:35.201613 14208 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3e4462c48b94d0784d26884c80b132f in term 1.
I20260926 20:45:35.203006 14357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } } }
I20260926 20:45:35.203539 14357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:35.214514 14283 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4 [term 1 FOLLOWER]: Refusing update from remote peer c3e4462c48b94d0784d26884c80b132f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:35.216120 14340 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:35.232059 14208 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6 [term 1 FOLLOWER]: Refusing update from remote peer c3e4462c48b94d0784d26884c80b132f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 20:45:35.233114 14340 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [LEADER]: Connected to new peer: Peer: permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 20:45:35.257634 14338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3e4462c48b94d0784d26884c80b132f. Latest consensus state: current_term: 1 leader_uuid: "c3e4462c48b94d0784d26884c80b132f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } } }
I20260926 20:45:35.258018 14338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:35.263165 14363 mvcc.cc:205] Tried to move back new op lower bound from 7333705872220643328 to 7333705871861628928. Current Snapshot: MvccSnapshot[applied={T|T < 7333705872220643328}]
I20260926 20:45:35.264870 14338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3e4462c48b94d0784d26884c80b132f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } } }
I20260926 20:45:35.265169 14338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:35.269078 14356 mvcc.cc:205] Tried to move back new op lower bound from 7333705872220643328 to 7333705871861628928. Current Snapshot: MvccSnapshot[applied={T|T < 7333705872220643328}]
I20260926 20:45:35.270741 14357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3e4462c48b94d0784d26884c80b132f. Latest consensus state: current_term: 1 leader_uuid: "c3e4462c48b94d0784d26884c80b132f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } } }
I20260926 20:45:35.271068 14357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:35.279913 14364 mvcc.cc:205] Tried to move back new op lower bound from 7333705872220643328 to 7333705871861628928. Current Snapshot: MvccSnapshot[applied={T|T < 7333705872220643328}]
I20260926 20:45:35.282785 14340 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3e4462c48b94d0784d26884c80b132f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } } }
I20260926 20:45:35.283095 14340 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:35.283370 14340 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3e4462c48b94d0784d26884c80b132f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } } }
I20260926 20:45:35.283558 14340 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f [sys.catalog]: This master's current role is: LEADER
I20260926 20:45:35.284435 14354 catalog_manager.cc:1347] Generated new cluster ID: 0a896c711d0f4e9fa782c0dc3796a7a0
I20260926 20:45:35.284534 14354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 20:45:35.308462 14357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3e4462c48b94d0784d26884c80b132f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e4462c48b94d0784d26884c80b132f" member_type: VOTER last_known_addr { host: "127.23.44.254" port: 44127 } } peers { permanent_uuid: "362b603c03754702aa46a82ff2322aa6" member_type: VOTER last_known_addr { host: "127.23.44.253" port: 37713 } } peers { permanent_uuid: "1a78bd4b1b1b47b9ba566664d92508f4" member_type: VOTER last_known_addr { host: "127.23.44.252" port: 38463 } } }
I20260926 20:45:35.308807 14357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 20:45:35.335647 14354 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 20:45:35.337755 14354 catalog_manager.cc:1504] Loading token signing keys...
I20260926 20:45:35.359838 14375 catalog_manager.cc:1259] Loaded cluster ID: 0a896c711d0f4e9fa782c0dc3796a7a0
I20260926 20:45:35.359965 14375 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6: loading cluster ID for follower catalog manager: success
I20260926 20:45:35.363873 14375 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6: acquiring CA information for follower catalog manager: success
I20260926 20:45:35.364117 14354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3e4462c48b94d0784d26884c80b132f: Generated new TSK 0
I20260926 20:45:35.365257 14354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 20:45:35.378064 14375 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 362b603c03754702aa46a82ff2322aa6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 20:45:35.522105 14314 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:35.496904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39454 (local address 127.23.44.254:44127)
0926 20:45:35.497096 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.497102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.519527 (+ 22425us) server_negotiation.cc:407] Connection header received
0926 20:45:35.520387 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.520398 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.520492 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.520592 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:35.521811 (+ 1219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39454: BlockingRecv error: recv got EOF from 127.0.0.1:39454 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260926 20:45:35.527043 14311 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:35.503852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50744 (local address 127.23.44.252:38463)
0926 20:45:35.504059 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.504077 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.524355 (+ 20278us) server_negotiation.cc:407] Connection header received
0926 20:45:35.525346 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.525352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.525447 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.525524 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:35.526773 (+ 1249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50744: BlockingRecv error: recv got EOF from 127.0.0.1:50744 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260926 20:45:35.527055 14315 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:35.518559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34188 (local address 127.23.44.253:37713)
0926 20:45:35.518781 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.518789 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.525078 (+ 6289us) server_negotiation.cc:407] Connection header received
0926 20:45:35.525346 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.525352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.525481 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.525550 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:35.526804 (+ 1254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34188: BlockingRecv error: recv got EOF from 127.0.0.1:34188 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 20:45:35.574774 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:35.558150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34204 (local address 127.23.44.253:37713)
0926 20:45:35.558510 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.558526 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.566224 (+ 7698us) server_negotiation.cc:407] Connection header received
0926 20:45:35.566690 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.566699 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.566968 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.567162 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:35.568108 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.569412 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.571527 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.572220 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.572324 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:35.573131 (+ 807us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:35.573883 (+ 752us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:35.574006 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:35.574272 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 20:45:35.574375 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
W20260926 20:45:35.625975 14311 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:35.612465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50760 (local address 127.23.44.252:38463)
0926 20:45:35.612722 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.612730 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.623604 (+ 10874us) server_negotiation.cc:407] Connection header received
0926 20:45:35.624221 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.624228 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.624375 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.624465 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:35.625715 (+ 1250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50760: BlockingRecv error: recv got EOF from 127.0.0.1:50760 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260926 20:45:35.678649 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:35.662752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34220 (local address 127.23.44.253:37713)
0926 20:45:35.662967 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.662975 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.670906 (+ 7931us) server_negotiation.cc:407] Connection header received
0926 20:45:35.671815 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.671825 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.671952 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.672041 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:35.672901 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.674026 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.676138 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.676869 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.676990 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:35.677684 (+ 694us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:35.677847 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:35.677918 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:35.678209 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 20:45:35.678310 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260926 20:45:35.727718 14311 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:35.713888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50764 (local address 127.23.44.252:38463)
0926 20:45:35.714094 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.714102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.725461 (+ 11359us) server_negotiation.cc:407] Connection header received
0926 20:45:35.726096 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.726102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.726218 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.726306 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:35.727459 (+ 1153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50764: BlockingRecv error: recv got EOF from 127.0.0.1:50764 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 20:45:35.793217 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:35.779099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34236 (local address 127.23.44.253:37713)
0926 20:45:35.779408 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.779416 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.785273 (+ 5857us) server_negotiation.cc:407] Connection header received
0926 20:45:35.785886 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.785892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.785999 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.786071 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:35.786844 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.788163 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.790390 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.791408 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.791655 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:35.792224 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:35.792408 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:35.792481 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:35.792771 (+ 290us) server_negotiation.cc:299] Negotiation successful
0926 20:45:35.792879 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260926 20:45:35.858376 14311 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:35.847393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50776 (local address 127.23.44.252:38463)
0926 20:45:35.847599 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.847605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.852405 (+ 4800us) server_negotiation.cc:407] Connection header received
0926 20:45:35.856896 (+ 4491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.856903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.857005 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.857092 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:35.858153 (+ 1061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50776: BlockingRecv error: recv got EOF from 127.0.0.1:50776 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 20:45:35.887773 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:35.877553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34246 (local address 127.23.44.253:37713)
0926 20:45:35.877803 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.877813 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.880418 (+ 2605us) server_negotiation.cc:407] Connection header received
0926 20:45:35.880655 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.880663 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.880798 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.880929 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:35.881473 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.882888 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.885075 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.885810 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.885962 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:35.886734 (+ 772us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:35.886919 (+ 185us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:35.887030 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:35.887231 (+ 201us) server_negotiation.cc:299] Negotiation successful
0926 20:45:35.887339 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260926 20:45:35.898152 14311 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:35.895782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50788 (local address 127.23.44.252:38463)
0926 20:45:35.896035 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.896042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.896322 (+ 280us) server_negotiation.cc:407] Connection header received
0926 20:45:35.896543 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.896574 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.896688 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.896788 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:35.897864 (+ 1076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50788: BlockingRecv error: recv got EOF from 127.0.0.1:50788 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
I20260926 20:45:35.918148 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:35.910700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.23.44.253:37713)
0926 20:45:35.910920 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.910927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.911490 (+ 563us) server_negotiation.cc:407] Connection header received
0926 20:45:35.911739 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.911746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.911877 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.911976 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:35.912488 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.913460 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.915683 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.916187 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.916292 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:35.916868 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:35.916978 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:35.917054 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:35.917486 (+ 432us) server_negotiation.cc:299] Negotiation successful
0926 20:45:35.917636 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260926 20:45:35.928077 14311 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:35.925877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50792 (local address 127.23.44.252:38463)
0926 20:45:35.926104 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.926110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.926514 (+ 404us) server_negotiation.cc:407] Connection header received
0926 20:45:35.926742 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.926746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.926821 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.926913 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:35.927834 (+ 921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50792: BlockingRecv error: recv got EOF from 127.0.0.1:50792 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 20:45:35.955152 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:35.947917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34262 (local address 127.23.44.253:37713)
0926 20:45:35.948113 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.948120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.948754 (+ 634us) server_negotiation.cc:407] Connection header received
0926 20:45:35.948981 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.948997 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.949099 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.949208 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:35.949709 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.950467 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.952831 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:35.953311 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:35.953416 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:35.954090 (+ 674us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:35.954224 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:35.954310 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:35.954516 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 20:45:35.954593 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260926 20:45:35.965600 14311 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:35.963163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50800 (local address 127.23.44.252:38463)
0926 20:45:35.963458 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:35.963466 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:35.964012 (+ 546us) server_negotiation.cc:407] Connection header received
0926 20:45:35.964212 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:35.964219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:35.964322 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:35.964446 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:35.965336 (+ 890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50800: BlockingRecv error: recv got EOF from 127.0.0.1:50800 (error 108)
Metrics: {"server-negotiator.queue_time_us":150}
I20260926 20:45:36.007754 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:36.001657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34268 (local address 127.23.44.253:37713)
0926 20:45:36.002054 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:36.002067 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:36.002739 (+ 672us) server_negotiation.cc:407] Connection header received
0926 20:45:36.002976 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:36.002980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:36.003053 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:36.003147 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:36.003603 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.004504 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.005969 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.006426 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.006529 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:36.006971 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:36.007099 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:36.007174 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:36.007328 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 20:45:36.007400 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
W20260926 20:45:36.016496 14311 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:36.014369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50812 (local address 127.23.44.252:38463)
0926 20:45:36.014617 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:36.014625 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:36.014938 (+ 313us) server_negotiation.cc:407] Connection header received
0926 20:45:36.015180 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:36.015184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:36.015277 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:36.015403 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:36.016222 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50812: BlockingRecv error: recv got EOF from 127.0.0.1:50812 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 20:45:36.090459 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:36.084156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34276 (local address 127.23.44.253:37713)
0926 20:45:36.084401 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:36.084406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:36.085021 (+ 615us) server_negotiation.cc:407] Connection header received
0926 20:45:36.085233 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:36.085238 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:36.085337 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:36.085445 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:36.086009 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.086786 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.088460 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.089048 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.089166 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:36.089730 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:36.089854 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:36.089943 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:36.090096 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 20:45:36.090170 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260926 20:45:36.100211 14311 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 20:45:36.097527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50822 (local address 127.23.44.252:38463)
0926 20:45:36.097733 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:36.097738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:36.098496 (+ 758us) server_negotiation.cc:407] Connection header received
0926 20:45:36.098648 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:36.098652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:36.098742 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:36.098839 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 20:45:36.099952 (+ 1113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50822: BlockingRecv error: recv got EOF from 127.0.0.1:50822 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 20:45:36.126753 14351 catalog_manager.cc:1259] Loaded cluster ID: 0a896c711d0f4e9fa782c0dc3796a7a0
I20260926 20:45:36.126854 14351 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4: loading cluster ID for follower catalog manager: success
I20260926 20:45:36.129840 14351 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4: acquiring CA information for follower catalog manager: success
I20260926 20:45:36.131963 14351 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a78bd4b1b1b47b9ba566664d92508f4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 20:45:36.237680 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:36.232209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34280 (local address 127.23.44.253:37713)
0926 20:45:36.232458 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:36.232468 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:36.232965 (+ 497us) server_negotiation.cc:407] Connection header received
0926 20:45:36.233131 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:36.233135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:36.233213 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:36.233305 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:36.233716 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.234506 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.235891 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.236342 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.236446 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:36.236863 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:36.236968 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:36.237041 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:36.237251 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 20:45:36.237319 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:45:36.252665 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:36.244989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50824 (local address 127.23.44.252:38463)
0926 20:45:36.245199 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:36.245205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:36.245629 (+ 424us) server_negotiation.cc:407] Connection header received
0926 20:45:36.246003 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:36.246009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:36.246232 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:36.246361 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:36.247080 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.247802 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.249751 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.250290 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.250433 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:36.251141 (+ 708us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:36.251767 (+ 626us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:36.251892 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:36.252123 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 20:45:36.252261 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 20:45:36.269439 14315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:36.260029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34292 (local address 127.23.44.253:37713)
0926 20:45:36.260226 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:36.260234 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:36.261459 (+ 1225us) server_negotiation.cc:407] Connection header received
0926 20:45:36.261730 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:36.261739 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:36.261870 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:36.261968 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:36.263016 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.263929 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.266664 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.267325 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.267485 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:36.268486 (+ 1001us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:36.268712 (+ 226us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:36.268811 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:36.268946 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 20:45:36.269054 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 20:45:36.269783 14311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:36.259850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50826 (local address 127.23.44.252:38463)
0926 20:45:36.260033 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:36.260039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:36.262444 (+ 2405us) server_negotiation.cc:407] Connection header received
0926 20:45:36.262658 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:36.262662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:36.262760 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:36.262845 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:36.264209 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.265540 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.267440 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.268067 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.268234 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:36.268775 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:36.268930 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:36.269024 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:36.269255 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 20:45:36.269352 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 20:45:36.270134 14482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 20:45:36.259762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.23.44.254:44127)
0926 20:45:36.262153 (+ 2391us) server_negotiation.cc:206] Beginning negotiation
0926 20:45:36.262160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 20:45:36.262182 (+ 22us) server_negotiation.cc:407] Connection header received
0926 20:45:36.262310 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 20:45:36.262316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 20:45:36.262557 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 20:45:36.262775 (+ 218us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 20:45:36.263666 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.264888 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.267251 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 20:45:36.267729 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 20:45:36.267824 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 20:45:36.268665 (+ 841us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 20:45:36.269176 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 20:45:36.269275 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 20:45:36.269736 (+ 461us) server_negotiation.cc:299] Negotiation successful
0926 20:45:36.269832 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2227,"thread_start_us":2070,"threads_started":1}
I20260926 20:45:36.274873 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 14092
I20260926 20:45:36.276678 14154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:36.392176 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 14092
W20260926 20:45:36.414395 14257 connection.cc:535] server connection from 127.0.0.1:59472 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 20:45:36.414634 14183 connection.cc:535] server connection from 127.0.0.1:36442 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 20:45:36.414882 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 14166
I20260926 20:45:36.416069 14228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:36.524010 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 14166
I20260926 20:45:36.546291 23731 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu pid 14241
I20260926 20:45:36.547528 14303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 20:45:36.665666 23731 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO4QBsv/build/asan/bin/kudu with pid 14241
2026-09-26T20:45:36Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4801 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9933 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (385008 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST